ir 


AD-A051  544 


UNCLASSIFIED 


NORTH  CAROLINA  UNIV  AT  CHAPEL  HILL  PSYCHOMETRIC  LAB  F/6  5/10 

HON  SHALL  NE  STUDY  INDIVIDUAL  DIFFERENCES  IN  C06NITIVE  ABILITIE— ETC (U) 
JAN  78  J B CARROLL  N00014-77-C-0722 


TR-1 


NL 


ADA051544 


HOW  SHALL  WE  STUDY  INDIVIDUAL  DIFFERENCES  IN  COGNITIVE 
ABILITIES? — METHODOLOGICAL  AND  THEORETICAL  PERSPECTIVES 


John  B.  Carroll 


University  of  North  Carolina  at  Chapel  Hill 


January  1978 


Technical  Report  No.  1 


D D C 


MAR  21  19(78 


B 


Sponsored  by 

Personnel  & Training  Research  Programs 
Psychological  Sciences  Division 
Office  of  Naval  Research 
Arlington,  Virginia 

Contract  No.  NOOO 14- 7 7-C-O 722  , NR  150-406 


Approved  for  public  release;  distribution  unlimited 

Reproduction  in  whole  or  in  part  is  permitted 
for  any  purpose  of  the  United  States  Government. 


THE  L.  L.  THURSTONE 
PSYCHOMETRIC  LABORATORY 
UNIVERSITY  OF  NORTH  CAROLINA 


CHAPEL  HILL,  N.  C. 
27514 


SECURITY  CLASSIFICATION  OF  THIS  PACE  ClWiwi  Datm  Bnlarad:  


4 -irtk.,  aA/'a  READ  INSTRUCTIONS 

REPORT  DOCUMEHTATION  PAGE before  completing  form 


number  - - - govt  accession  NO.l  3-  RECIPIKNT’S  CATALOG  NUMBCP 


‘How  ^hall  ^tudy  individual  differences 
Tn  cognitive  jl)iliti?s4l--Wethodological  and 


theoretical  oerspectives ^ 


S.  TYPE  OF  report  ft  PERIOD  COVERED 


Technical/4(e|i«vt. 


(i. 


7.  AUTmORC«J 


John  B. /Carroll 


9.  PERFORMING  organization  NAME  AND  ADDRESS 

1 L.  L.  Thurstone  Psychometric  Laboratory 
University  of  North  Carolina 
Chapel  Hill,  NC  27514 

11.  CONTROLLING  OFFICE  NAME  AND  ADDRESS 

Personnel  and  Training  Research  Programs  ( 

Office  of  Naval  Research  (Code  458) 

Arlington,  VA  22217 

iA.  MONITORING  AGENCY  NAME  » AODRESSCK  dlllatani  Irom  ConlrolllnE  Otttca) 


\ IS.  DISTRIBUTION  STATEMENT  (ol  thia  Raporl) 


S.  PERFORMING  ORG.  REPORT  NUMBER 

No.  1 __ 


« eONTRACT  QB.g»AMT  NMMBERf.) 

n^l4_77-C^7^ 


10.  PROGRV^lLUHI.fJIUJLUI. 
APE  A 

RK  045  :04; 


NUMBEH  UF  PAULJ  ' Z — 7 

57  ij. 

IS.  SECURITY  CLASS,  (ol  Ihlat^otl) 

Unclassified 


ISa.  DECLASSIFICATION/ DOWNGRADING 
SCHEDULE 


Approved  for  public  release;  distribution  unlimited. 


I 17.  OtSTRtBUTtON  STATEMENT  (oi  thm  mb»tfaet  tiffa  In  Block  30,  If  dtfforont  from  Roport) 


10.  SUPPLEMENTARY  NOTES 


I to.  KEyWOBO$(ConOnu0onrovot»ooidmlittoeooomrtnd  Idmntily  by  block  numbor) 


Abilities  Intelligence 

Factor  analysis  Individual  differences 

Cognition 

Cognitive  abilities 

20.  abstract  ^DonUnua  on  raaataa  alda  If  nacaaaaay  and  Idantltr  by  block  numbar)  / ^ / , L 

/V  Selectedv.studies  in  ^reocwt ' reacoreff  literature  «««/examined  with  regard  to: 
CA)  their  methodological  adequacy  in  establishing  dimensions  of  individual  I 
differences  in  information-processing  abilitiesf  and  (j!^)  the  theoretical 
problems  involved  in  inferring  psychological  processes  from^iqdi^idual 
difference  dimensions.  Many  methodological  inadequacies  ^rd'^noted,  including 
use  of  small  N's,  questionable  procedures  in  establishing  variables  (e.g.,  use 
of  raw  difference  scores),  improper  or  inadequate  use  of  factor  analysis  and 

DD  I 1473  EDITION  OF  I NOV  68  1$  OBSOLETE  ) IT  1 ’ f ' H ^ 

SJN  ^02-  LF-  OU-  66^  V^ECURlTY  CLASSIFICATION  OF  THIS  PAGE  (Whan  Data  BnMt«« 

<^90  is'O 


Unclassified 

secuniTY  CLASSIFICATION  OF  THIS  NAOC  rWhwi  Data  Bnftmd) 


pother  multivariate  statistical  techniques,  and  poor  presentation  of  results. 
On  the  whole,  little  progress  has  been  made  thus  far  in  identifying  psychol- 
ogical processes  through  research  in  individual  differences,  even  though 
this  research  approach  is  viable  and  potentially  useful.  Serious  theoretical 
difficulties  arise  in  attempting  to  infer  the  nature  and  operation  of  psychol 
ogical  processes  merely  from  the  identification  of  individual  difference 
trait  dimensions.  Promising  research,  however,  is  represented  by  studies  in 
which  an  effort  is  made  to  analyze  tasks  into  their  components,  to  vary  task 
characteristics,  and/or  to  consider  the  strategies  that  individuals  can 
employ  in  performing  the  tasks. 


-X 


ACCeSSlOW  for 


NTIS  White  Section 

DOC  Buff  Section 

UNANNOUNCED  □ 

JUSTIFICATION  


□ 


n 

KRBUl 

im/nYnnmr  cooes 

IWet  AVAIL  and/or  SPECIaII 

/I 

S/N  0102-  LF-  0I4-e«01 


Unclassified 

SCCUNITV  CLAUIFICATION  of  THiS  PAOCfWhwi  Dm*  Bnitnd) 


HOW  SHALL  WE  STUDY  INDIVIDUAL  DIFFERENCES  IN  COGNITIVE  ABILITIES?— 
METHODOLOGICAL  AND  THEORETICAL  PERSPECTIVES 


John  B.  Carroll 


University  of  North  Carolina  at  Chapel  Hill 


January  1978 


Approved  for  public  release;  distribution  unlimited. 
Reproduction  in  whole  or  in  part  is  permitted 
for  any  purpose  of  the  United  States  Government. 


1 


1 


How  shall  we  study  individual  diffe~ences  in  cognitive  abilities?-- 
Methodological  and  theoretical  perspectives 


"How  shall  we  study  individual  differences?"  This  was  the 
question  considered  by  S.S.  Sargent  (1942)  in  a paper  of  that 
title  published  in  the  Psychological  Review  in  1942.  It  is  a 
question  that  needs  to  be  asked  again  and  again,  for  it  is  one 
that  has  no  final  answer.  Certainly  Sargent  was  not  the  first  to 
have  raised  it — the  problem  goes  back  to  the  time  of  Galton,  James 
McKeen  Cattell,  Edward  L.  Thorndike  and  other  founders  of  psycho- 
metrics. The  particular  concerns  addressed  by  Sargent  in  his  1942 
article,  however,  were  somewhat  novel  at  the  time,  and  they  are 
particularly  pertinent  to  our  current  interest  in  the  analysis  of 
"intelligence"  in  the  light  of  information-processing  theories. 
Sargent  was  pointing  out  that 

...quantitative  approaches  do  not  give  an  accurate  picture  of 
individual  differences.  ...  quantitative  treatment,  per  se , 
does  not  describe  the  methods  of  work  used  by  a subject  as 
he  performs  a task;... it  does  not  depict  adequately  the  pat- 
tern of  behavioral  processes  involved;  [and]  preoccupation 
with  quantitative  method  causes  one  to  lose  sight  of  Important 
aspects  of  individual  personalities  and  therefore  of  differences 
between  personalities."  (Sargent,  1942,  p.  171). 
"Information-processing"  had  not  become  the  catch-phrase  it  is  today. 
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but  wo  can  assume  that  Sargent  was  thinking  of  what  we  now  rail 
information-processing  when  he  wrote  about  "methods  of  work" 
and  "behavioral  processes."  And  when  he  mentioned  "important 
aspects  of  personalities,"  perhaps  he  was  thinking  of  what  we 
would  now  call  "cognitive  styles."  In  this  paper  1 will  not  con- 
cern myself  with  cognitive  styles,  but  I will  address  the  question 
of  how  we  can  get  at  the  methods  of  work  and  the  behavioral  pro- 
cesses that  presumably  underlie  individual  differences  in  cognitive 
abilities . 

Let  us  remind  ourselves  that  Thurstone,  in  his  use  of  factor 
analysis,  was  much  concerned  with  the  explanation  of  mental  abili- 
ties in  terms  of  psychological  processes.  This  will  be  evident 
from  the  most  casual  examination  of  his  attempts  to  infer  the  psy- 
chological meanings  of  the  factors  he  identified.  But  Thurstone 
was  dissatisfied  with  purely  intuitive  interpretations  of  factors. 

At  one  point  he  stated  that  his  preference  would  be  "to  head  as 
soon  as  possible  to  direct  forms  of  laboratory  experiments  in  terms 
of  which  the  primary  factors  may  eventually  be  better  understood" 

( thurstone,  1940,  p.  204). 

Within  the  last  few  years,  psychologists  have  gone  into  the 
laboratory  in  droves  to  look  at  possible  relations  between  mental 
abilities  and  variables  in  the  kinds  of  experimental  tasks  that  are 
characteristically  studied  in  cognitive  psychology.  How  are  these 
psychologists  faring  in  arriving  at  a better  understanding  of  mental 
abilities? 

Currently,  I am  engaged  in  preparing  a review  of  individual 
difference  research  for  the  1979  volume  of  the  Annual  Review  of 


r 


3 


Psychology . This  work  has  prompted  me  to  assemble  a large  parcel 
of  recent  psychological  research  literature  pertinent  to  an  informa- 
tion-processing view  of  mental  abilities.  Surveying  and  carefully 
examining  this  literature  has  caused  me  to  conclude  that  little 
progress  has  been  made  thus  far  in  understanding  mental  abilities 
• in  terms  of  processes.  It  can  be  argued,  to  be  sure,  that  there 

has  been  some  success  in  identifying  psychological  processes,  but 

I 

; the  interpretation  of  these  processes  often  stands  or  falls  depend- 

• ing  upon  whether  one  can  accept  the  information-processing  models 

upon  which  the  identification  of  a particular  process  is  based. 

Further,  the  experimental  identification  of  a process  often  depends 
chiefly  upon  the  finding  of  individual  differences  in  the  parameters 
of  the  process,  which  in  turn  are  frequently  quite  specific  to  that 
process.  This  has  led,  in  effect,  to  the  identification  of  a whole 
new  series  of  individual  "traits"  that  are  little  related  to  the 
mental  abilities  isolated  in  classical  psychometric  studies.  Even 
if  the  relations  are  found  to  be  of  substantial  magnitude,  it  is  not 
very  revealing  or  informative  merely  to  establish  correspondences 
between  traits  and  processes  that  are  defined  largely  on  the  basis 
of  those  traits.  There  is  an  obvious  danger  of  circularity  in  all 
this. 

, In  what  follows,  I want  to  expand  this  point  of  view  and  offer 

some  thoughts  on  how  we  can  avoid  circularity  and  make  independent 
determinations  of  processes  and  individual  difference  variables. 

In  preparing  my  review,  however,  I have  been  disturbed  to  discover 
that  information-processing  psychologists,  and  even  some  psychome- 
trlcally-oriented  psychologists,  have  in  many  cases  misapplied  tra- 
ditional psychometric  methodologies  and  have  drawn  Inferences  and 

i 
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conclusions  that  I believe  are  unjustified  or  at  least  questionable. 
Before  1 discuss  problems  of  studying  psychological  processes,  I 
aim  to  clear  the  ground  by  offering  an  extensive  critique  of  the 
statistical  methodologies  that  are  now  being  utilized  in  the  attempt 
to  study  individual  differences  in  mental  abilities  from  an  Informa- 
tion-processing point  of  view. 

Statistical  Methodologies  in  Individual  Differences  Research 

For  many  years,  statistical  methodology  in  studying  individual 
differences  has  been  essentially  correlational,  resting  usually  on 
bivariate  or  multivariate  linear  models.  Perhaps  it  is  inevitable 
that  this  methodology  is  correlational,  for  a first  approach  to  the 
study  of  individual  differences  is  to  examine  the  generality  of  those 
differences  over  sets  of  observed  variables,  leaving  aside  the  ef- 
fects of  differential  treatments  or  manipulated  conditions.  Since 
the  time  of  Spearman  and  Pearson,  it  has  been  assumed  that  if  two 
measures  taken  on  a given  sample  are  significantly  associated, 
they  may  be  regarded  as  o some  extent  measuring  "the  same  thing"; 
further,  that  if  two  acceptably  reliable  measures  are  not  signifi- 
cantly associated,  they  are  measuring  "different  things."  If  the 
observations  are  taken  over  a sample  of  persons,  these  "same"  and 
"different"  things  are  often  assumed  to  represent  attributes, 
characteristics,  or  "traits"  of  the  persons.  Such  an  assumption 
lies  at  the  base  of  factor-analytic  methodology,  which  is  claimed 
to  permit  a detailed  analysis  of  the  multiple  determination  of 
observed  variables  by  inferred  latent  traits.  The  theory  of  mental 
tests  makes  appeal  to  such  an  assumption  in  its  postulation  of 
"true  scores"  on  latent  traits  that  underlie  observed  measurements. 
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A persisting  source  of  controversy  in  psychometrics  is  over 
whether  such  latent  traits  are  mere  statistical  artifacts  or, 
rather,  represent  real  entitles  in  the  makeup  of  individuals. 

Let  us  for  the  moment  leave  this  controversy  aside  and  focus  on 
purely  statistical  aspects  of  individual  difference  methodology. 

Simple  correlations 

The  literature  of  research  in  individual  differences  provides 
countless  examples  of  the  use  of  simple  correlation  to  support  the 
claim  of  a common  element  existing  between  two  observed  variables. 

It  is  tempting  to  assume  that  this  common  source  of  variance  Is  a 
single  entity.  It  is  often  forgotten  that  a single  correlation 
could  reflect  the  common  operation  ( pr  lack  of  operation)  of  nume- 
rous sources  of  variance — sources  of  variance  that  could,  presuma- 
bly, be  teased  out  only  by  some  appropriate  multivariate  design. 

I will  cite  and  comment  on  two  examples  of  the  use  of  simple, 
zero-order  correlations  in  current  literature,  chosen  because  they 
well  illustrate  certain  points  I wish  to  make. 

Cohen  and  Sandberg  (1977)  were  concerned  with  the  relation  be- 
tween Intelligence  and  short-term  memory  (STM).  Their  method  was, 
essentially,  to  dissect  the  supraspan  memory  test  into  certain  com- 
ponents, searching  for  those  one  or  more  components  that  showed 
correlations  with  IQ  as  measured  by  a certain  intelligence  scale 
constructed  for  Swedish  children.  A typical  finding  was  a correlation 
of  .68,  highly  significant,  between  IQ  and  performance  on  probed  re- 
call of  the  last  three  digits  of  a 9-digit  sequence,  but  a correlation 
of  .06,  not  significant  with  N *■  38,  between  IQ  and  probed  recall  of 
the  first  three  digits.  Actually,  their  paper  includes  replication 
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of  this  finding  on  a number  of  groups  in  a variety  of  conditions, 
and  1 am  not  doubting  its  reliability.  I merely  wish  to  point 
out  that  (except  in  plotting  trends  by  score  categories  to  demon- 
strate "continuity"  of  the  function)  they  did  not  go  beyond  the 
use  of  simple  correlations  to  explore  further  implications.  The 
significant  correlations  could  have  reflected  a number  of  different 
sources  of  variance;  since  the  "IQ"  test  actually  consisted  of 
six  subtests,  it  might  have  been  fruitful  to  inquire  whether  the 
correlations  were  higher  for  certain  subtests  than  others.  One 
could  question  whether  the  subtests  are  truly  measures  of  IQ;  as 
described,  the  subtests  were  "designed  to  measure  verbal  perform- 
ance (synonyms  and  antonyms),  abstract-logical  ( inductive)  reason- 
ing, and  spatial  performance"  (Cohen  & Sandberg,  1977,  p.  538). 

Some  of  the  subtests,  therefore,  could  have  reflected  special 
kinds  of  learning. 

My  second  example  comes  from  a series  of  investigations  by 
Hock  and  various  coauthors.  In  the  first  of  these.  Hock  (1973) 
reported  a correlation  of  .60,  £ < .05,  between  what  he  called  a 
"symmetry  effect"  and  a "rotation  effect,"  in  a study  of  individ- 
uals' reaction  times  (RTs)ln  a same-different  comparison  task  in- 
volving dot  patterns.  Hock  interpreted  this  result  as  reflecting 
individual  differences  in  modes  of  processing  the  stimuli.  In- 
dividuals whose  RTs  were  affected  by  asymmetry  and  by  rotation  of 
the  stimuli  were  said  to  be  using  "structural  processing,"  whereas 
individuals  whose  RTs  were  not  affected  by  these  variables  were 
thought  to  be  using  an  "analytic"  mode  of  processing.  It  must  be 


pointed  out,  however,  that  the  reported  correlation  was  based  on 
only  twelve  cases;  furthermore,  if  we  examine  the  scatterplot  of 
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scores  on  the  two  variables,  it  Is  evident  that  the  correlation 
arises  mainly  from  the  presence  of  one  or  two  strikingly  outlying 
cases;  omitting  the  most  extreme  of  these  reduces  the  correlation 
to  .34,  and  omitting  the  next  most  extreme  reduces  it  to  .28, 
neither  value  being  significant  with  p < .05.  Hock  himself  did 
not  comment  on  the  unusual  distribution  of  cases  in  the  scatter- 
plot  ( or  in  the  underlying  distributions),  but  he  did  recognize 
that  the  correlation  might  have  been  due  to  an  "artifact  of  per- 
formance level";  partialing  out  mean  RTs  to  familiar,  symmetrical 
patterns  as  measures  of  overall  performance  level,  the  resulting 
correlation  between  the  symmetry  effect  and  the  rotation  effect 
was  still  .60,  according  to  Hock  (1973,  fn.  4).  However,  both 
these  effects  were  measured  as  di f f erences  between  mean  RTs;  one 
can  question  the  meaning  of  a partial  correlation  between  difference 
variables  when  the  partialed-out  variable  is  a variable  that  enters 
into  the  computation  of  the  differences. 

It  should  be  observed,  also,  that  RTs  are  themselves  notorious- 
ly unreliable  and  variable,  and  their  distributions  are  often  quite 
skewed  and  loaded  with  outliers.  My  experience  with  RTs  has  been 
that  it  is  wise  to  transform  them  before  taking  means;  my  preference 
(which  for  lack  of  space  I will  not  attempt  to  justify  here)  is  to 
use  the  reciprocal  transformation,  and  to  report  mean  recipro^cals 
or  the  inverse  of  the  mean  reciprocal  (in  effect,  the  harmonic  mean). 
(See  Wainer,  1977,  for  further  comment  on  this  matter.)  When  dif- 
ferences between  means  are  taken,  and  especially  when  this  is  done 
for  Individual  Ss , one  is  creating  variables  whose  reliability  must 
be  carefully  examined.  Even  though  in  his  later  studies  (e.g.. 


Hock,  Gordon,  & Corcoran,  1976;  Hock  & Ross,  1975)  Hock  used  slightly 
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larger  sample  sizes  in  attempting  to  support  his  claim  of  an 
individual  difference  variable  contrasting  "structural"  and 
"analytical"  modes  of  processing,  there  are  persisting  methodolog- 
ical problems  of  the  types  mentioned  here.  Furthermore,  in  none  of 
these  studies  has  Hock  attempted  to  identify  an  independent  measure 
of  the  individual  difference  variable  he  claims. 

Factor-analytic  methodology 

Factor  analysis  has  been  the  classical  method  of  choice  in  the 
study  of  individual  differences,  and  increasingly,  experimental 
psychologists  are  turning  to  its  use.  Despite  its  many  virtues, 
factor  analysis  is  a very  tricky  technique;  in  some  ways  it  depends 
more  on  art  than  science,  that  is,  more  on  intuition  and  judgment 
than  on  formal  rules  of  procedure.  People  who  do  factor  analysis 
by  uncritical  use  of  programs  in  computer  packages  run  the  risk  of 
making  fools  of  themselves.  One  can  even  be  misled  by  misspellings 
in  computer  programs;  I don't  know  how  many  times,  in  published 
literature,  that  I have  seen  principal  components  spelled  "principle 
components,"  presumably  because  several  widely-used  computer  pro- 
grams happen  to  spell  it  that  way  in  their  print-outs.  I assure 
you  that  the  correct  spelling  is  principal  (Hotelling,  1933).  But 
there  are  also  a host  of  methodological  problems  that  beset  the  unwary 
factor  analyst  . Many  of  these  were  discussed  in  an  article  by 
Thurstone,  "Current  issues  in  factor  analysis"  (Thurstone,  1940), 
but  apparently  this  article  is  seldom  read  any  more.  I will  cite 
some  of  the  problems  by  commenting  on  the  factor-analytic  method- 
ology used  by  Jarman  and  Das  ( 1977)  in  an  article  that  is  almost 
fresh  off  the  press  in  a new  and  hopefully  prestigious  journal.  If 
I single  out  this  stadyfor  comment,  I do  so  only  because  it  is  a 
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small,  concise  study  that  Is  easy  to  present  and  discuss.  Many 
of  my  remarks  could  equally  well  be  directed  to  various  other  re- 
cent studies  in  the  literature. 

Jarman  and  Das  were  concerned  with  establishing  an  "alterna- 
tive model  of  mental  abilities"  that  appeals  to  1 n f o rma t lon-p r oce s s- 
ing  theories  that  claim  information  can  be  processed  either  by 
"simultaneous  syntheses"  or  by  "successive  syntheses."  They  made 
separate  factor  analyses  of  seven  psychometric  and  experimental 
variables  obtained  on  60  4th-grade  boys  in  each  of  three  ranges  of 
IQ  as  determined  by  the  l.o r ge-Tho rn dike  Intelligence  Test--Low, 
Average!  "Normal"),  and  High.  Principal  component  analysis  with 
varimax  rotation  yielded  three  factors  for  the  Low  and  High  groups, 
and  two  factors  for  the  "Normal"  group.  In  each  case,  one  factor 
was  Identified  as  representing  the  operation  of  "simultaneous 
8 ynth e se s; " the  one  or  two  other  factors  were  identified  with  "suc- 
cessive syntheses"  and  speed,  or  some  combination  thereof.  Let  me 
make  a number  of  observations  about  the  methodology  and  presentation 
of  these  results. 

( 1)  The  small  N * s . Jarman  and  Das  state  (p.  154):  "The  se- 
lection of  a group  size  of  60  was  based  on  the  requirement  that  there 
be  a sufficiently  large  sample  to  perform  wlthin-group  principal  com- 
ponent and  common  factor  analyses."  Certainly  an  N of  60  is  a bare 
minimum  for  establishing  reliable  results.  It  is  better  than  the 
N's  of  around  20  to  40  that  are  being  used  by  many  experimenters  in 
the  individual  differences  field,  to  be  sure,  but  one  would  still 
wish  for  a large  sample  size,  I am  afraid  that  this  matter  of  sample 
size  is  going  to  plague  the  field  for  quite  a time,  for  the  kinds  of 
experimental  learning  or  performance  tasks  that  we  want  to  study  in 


10 


an  information-processing  mode  require  much  more  time  to  conduct 
or  administer  than  the  brief  group  paper-and-pencil  tests  that 
have  traditionally  been  used  in  psychometric  studies.  The  problem 
is  compounded  when  one  wants  to  study  a large  number  of  variables. 
Sometimes  the  number  of  variables  actually  approaches  or  even  ex- 
ceeds the  sample  sire  (e.g.,  Favero,  Dombrower,  Michael,  & Richards, 
1975).  As  a rule  of  thumb  (fo,  which  I can  give  a certain  justi- 
fication), I recommend  that  to  establish  m factors,  the  sample 
size  be  at  least  as  great  as  the  quantity  (2m  + 2—).  On  this  basis, 
Jarman  and  Das's  sample  sizes  were  large  enough  to  establish  some- 
thing like  5 factors,  but  they  didn't  have  enough  variables  to  do 
so.  This  leads  me  to  the  next  observation: 

(2)  The  small  number  of  variables.  Since  Jarman  and  Das  were 

interested  only  in  establishing  two  factors,  it  could  be  argued 
that  seven  variables  were  sufficient.  This  is,  for  example,  larger 
than  what  would  be  requije-d  by  Thurstone's  ( 1947,  p.  293)  criterion 
n [(2m  + 1)  + (8m  + 1)^  ]/2  for  the  minimum  number  (n)  of 

variables  required  for  the  determination  of  m factors.  On  the  other 
hand,  experience  has  shown  that  restricting  oneself  to  small  numbers 
of  tests  in  a factor  battery  does  not  permit  the  kind  of  variation 
and  sampling  of  factor  domains  that  is  desirable  to  provide  persua- 
sive evidence  for  the  interpretation  of  any  factors  that  may  be 
found.  Certainly  Jarman  and  Das  would  be  encouraged  to  explore  the 
nature  of  their  factors  with  a wider  selection  of  variables. 

(3)  Failure  to  reflect  variables.  One  of  the  most  bothersome 
things,  I find,  in  the  inspection  of  factor  analytic  results  is 
authors'  failure  to  orient  all  their  variables  in  some  consistent 


direction,  i4e.,  preferably  with  the  positive  (algebraically  greater) 
side  being  toward  the  more  correct,  desirable,  fast,  efficient,  etc. 
kind  of  behavior.  In  Jarman  and  Das ' s factor  matrices,  we  find  a 
number  of  large  negative  values.  One  immediately  wonders  whetlier 

the  matrices  fail  to  exhibit  what  Thurstone  (1947,  p.  341)  called 
positive  manifold,  i.e.  a condition  where  all  the  loadings  are  posi- 
tive or  vanishingly  zero  after  rotation  for  simple  structure.  It 
turns  out,  in  the  Jarman  and  Das  data,  that  two  of  the  variables 
were  entered  into  the  correlation  matrices  in  what  I call  negative 
orientation,  i.e.  high  values  were  associated  with  error  or  slowness. 
For  one  of  the  tests  (Memory  for  Designs)  the  score  was  the  number 
of  errors,  and  for  another  test  (Word  Reading)  the  score  was  time 
for  the  subject  to  read  40  words.  If  I had  been  reporting  and  ana- 
lyzing these  data,  I would  have  replaced  the  error  score  by  a "number 
correct"  score,  and  I would  have  converted  the  time  score  to  a rate- 
of-per f ormance  score  (by  using  some  multiple  of  the  reciprocal  of 
the  time  score,  e.g.  words  per  minute).  (Usually,  such  a transforma- 
tion produces  a more  symmetrical  distribution.)  As  it  is,  one  can 
try  to  remedy  the  situation  only  by  reflecting  signs  in  selected 
rows  of  the  factor  matrices. 

All  this  is  mostly  a matter  of  nicety  and  clarity  in  presenta- 
tion: of  course  nothing  is  really  changed  in  the  results  (except 
when  one  reverses  orientation  by  making  a reciprocal  transformation, 
as  in  the  case  of  the  time  score).  The  problem  becomes  particularly 
acute  in  connection  with  difference  scores.  Authors  sometimes  fail 
to  report  the  direction  in  which  they  take  differences.  For  example, 
Lunneborg  (1977,  p.  311)  reported  a "Stroop  Difference"  score  as  "the 
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average  difference  In  'reading  times'  between  the  name  and  asterisk 
conditions."  I am  unaware  of  any  convention  whereby  such  a state- 
ment would  convey  whether  the  score  was  computed  as  (N  - A)  or  as 
(A  - N) , and  one  cannot  decide  which  it  was  on  the  basis  of  any 
other  statement  in  Lunneborg's  paper.  (From  Hunt,  Lunneborg,  and 
Lewis's  report  [1975]  on  the  same  data,  one  finds  that  the  Stroop 
difference  score  was  computed  as  [N  - A].)  Of  course,  sometimes 
one  is  not  able  to  assess  in  advance  how  a difference  score  is 
best  oriented,  but  this  matter  can  usually  be  decided  in  terms 
of  the  configuration  of  factor  loadings,  and  taken  care  of  at  the 
time  of  preparing  the  final  results. 

(A)  Failure  to  reflect  factors.  Here  is  another  matter  that 
1 find  bothersome,  though  not  really  wrong.  Frequently  we  see  factor 
matrices  with  most  of  their  large  loadings  negative.  Or  sometimes 
we  see  them  with  some  high  positive  loadings,  and  some  negative. 

Often  this  situation  arises  because  of  the  failure  to  reflect  vari- 
ables, as  just  mentioned.  But  even  after  reflecting  variables  ap- 
propropr lately , one  can  still  have  a large  number  of  negative  load- 
ings. Again,  one  Immediately  raises  the  question  of  a possibly 
non-positive  manifold.  In  nearly  every  instance  in  my  experience, 
the  large  number  of  negative  loadings  arises  simply  because  the  com- 
puter knows  nothing  about  positive  manifolds;  it  can  make  the  load- 
ings for  a factor  mostly  all  positive,  or  mostly  all  negative,  de- 
pending upon  certain  conditions  in  the  computation  of  eigenvectors 
or  in  the  process  of  analytic  rotation.  The  orientation  of  a factor 
is  entirely  arbitrary,  as  far  as  the  mathematics  is  concerned.  Re- 
gardless of  whether  a factor  vector  is  oriented  positively  or  negative- 
ly, it  will  make  the  same  contribution  to  the  reproduced  correlation 
matrix,  because  in  reproducing  the  correlation  matrix,  one  is 
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multiplying  entries  pairwise  within  a vector;  in  matrix  notation, 
these  multiplications  (for  uncorrelated  factors)  are  represented 
as  = FF’ , where  £ is  the  factor  matrix  (variables  X factors) 
and  is  the  reproduced  correlation  matrix.  The  situation  is 
quite  analogous  to  the  computation  of  a square  root,  which  can  be 
either  positive  or  negative:  computers  are  "trained"  to  report 
positive  square  roots,  normally,  and  they  can  be  "trained"  or  pro- 
gramed to  report  positively  oriented  factor  vectors.  I recommend 
to  all  authors  of  factor  analysis  computer  programs  that  they  pro- 
gram in  such  a way  as  to  change  all  the  signs  of  any  factor  vector 
( either  in  eigenvector  or  in  analytic  rotation  routines)  that  fails 
to  have  a positive  algebraic  sum.  Many  currently  available  computer 
package  programs  fail  to  do  this.  The  remedy,  short  of  changing 
these  programs,  is  to  change  the  signs  by  hand. 

When  appropriate  reflections  of  variables  and  factors  are  made 
for  the  Jarman  and  Das  matrices,  they  exhibit  generally  positive 
manifold.  Even  then,  the  matrices  are  of  doubtful  value  because  of 
another  unwise  procedure  of  analysis  that  these  authors  followed: 

(5)  Separate  factor  analyses  by  ability  strata.  As  noted, 
Jarman  and  Das  reported  separate  factor  matrices  for  three  groups 
defined  by  IQ.  To  be  sure,  IQ  was  not  one  of  the  variables  Included 
in  the  matrices,  although  one  of  the  variables  (Raven's  Progressive 
Matrices)  is  often  regarded  as  a measure  of  IQ,  and  Indeed  Jarman 
and  Das's  Table  2 (showing  means  of  all  variables  for  subgroups) 


suggests  that  all  the  variables  were  correlated  with  IQ  to  at  least 
an  appreciable  extent.  Now,  doing  separate  factor  analyses  by  ability 
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strata  is  a very  risky  procedure.  Obviously,  it  entails  restrictions 
of  range  and  the  consequent  attenuation  of  correlations.  It  can 
also  entail  the  creation  of  peculiar  distributions  of  variables, 
even  if  there  are  no  celling  or  floor  effects  in  the  tests  themselves, 
because  one  is  selecting  from  different  portions  of  approximately 
■ Gaussian  distributions.  These  peculiar  distributions  can  affect 

'I 

the  correlations  in  various  somewhat  unpredictable  ways,  as  I have 

!■ 

pointed  out  in  an  earlier  publication  (Carroll,  1961).  And,  of 
course,  the  sample  sizes  are  automatically  much  reduced,  with  con- 
sequent loss  of  statistical  power.  Jarman  and  Das  opted  for  the 
analyses  by  strata  on  the  basis  of  their  supposition  that  "dif- 
ferent levels  of  intelligence  ...  may  be  characterized  by  different 
uses  of  simultaneous  and  successive  syntheses  for  particular  tasks," 
and  their  statement  of  the  purpose  of  their  study  as  being  "to  iden- 
tify the  similarities  and  differences,  if  any,  in  the  employment  of 
simultaneous  and  successive  syntheses  by  groups  of  children  differ- 
ing in  IQ"  (Jarman  & Das,  1977,  p.  153).  Unfortunately,  because 
of  the  limitations  just  noted,  factor  analysis  by  ability  strata  is 
not  in  general  a sufficiently  reliable  and  effective  tool  to  inves- 
tigate hypotheses  concerning  differential  use  of  processes  at  dif- 
ferent levels  of  ability.  Such  hypotheses,  I would  suggest,  could 
better  be  Investigated  at  the  level  of  particular  correlations, 
e.g.t  by  testing  equality  of  regression  slopes  over  ability  groups 
for  particular  sets  of  variables,  or  by  using  contingency  tables 
and  other  non-parametric  techniques.  Possibly  Joreskog's  (1970) 
methods  of  covariance  structure  analysis  would  be  useful.  But  for 
a preliminary  evaluation  of  a set  of  data,  I would  recommend  factor 
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analysis  based  on  a single  group  pooled  from  the  several  strata, 
even  if  the  single  group  is  not  completely  representative  of  some 
population  because  of  gaps  in  the  distributions,  as  where,  for 
example,  "high"  and  "low"  tails  or  segments  of  some  stratification 
variable  are  pooled  (Hunt,  Frost,  & Lunneborg,  1973;  Hunt,  Lunneborg, 

& Lewis , 19  75)  . 

(6)  Use  of  principal  component  analysis.  On  this  matter  there 
is,  I acknowledge,  a difference  of  opinion  among  experts,  and  some 
will  say  that  it  doesn't  make  much  difference  what  factoring  method 
is  used.  Jarman  and  Das  used  a principal  components  model  (that  is, 

1 assume,  an  eigenvector  factoring  of  a correlation  matrix  with  unities 

(j  in  the  diagonal)  with  varimax  rotation,  "for  reasons  of  comparability 

to  previous  research"  (p.  161).  They  also  report  that  "high  corre- 
spondence" was  found  between  the  principal  components  analysis  and 
an  alpha  factor  analysis  that  was  also  computed.  There  could  indeed 
have  been  high  correspondence  in  patterns  of  results,  but  principal 
components  analysis  tends  to  yield  factor  loadings  that  are  consider- 
ably inflated  over  those  of  alpha  and  other  types  of  common  factor 
analysis,  leading  to  overgenerous  factor  interpretations.  I <"lnd 
principal  components  analysis  useful  chiefly  in  helping  to  decide 
on  the  number  of  factors  to  be  used  in  subsequent  communality  es- 
timations and  common  factor  analyses.  I much  prefer  some  form  of 
common  factor  analysis  that  avoids  the  intrusion  of  variance  uniquely 
associated  with  each  variable  into  the  common  factor  space. 

( 7)  Problems  of  factor  rotation;  orthogonal  vs.  oblique  factors. 
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Nevertheless,  if  one  pools  the  data  over  the  groups  ( as  I have 
done,  with  the  accuracy  permitted  by  what  these  authors  report, 
and  doing  tlic  pooling  on  the  basis  of  the  reported  group  means 
and  standard  deviations  and  the  correlations  reproduced  by  the 
reported  factor  matrices),  and  if  one  uses  a common  factor  analysis, 
the  data  are  satisfactorily  fit  by  two  factors.  A graphical  rota- 
tion of  these  factors  to  simple  structure  suggests  that  the  factors 
are  to  some  extent  correlated.  The  resulting  analysis  is  given 
as  Table  1,  which  also  shows  a Schmid-Leiman  (Schmid  & Lelman  1957) 


Table  1 about  here 

or thogonalization  of  the  data  in  such  a way  as  to  exhibit  a "general" 
factor  and  two  group  factors.  (I  do  not  mean  to  identify  this 
general  factor  with  Spearman's  "g",  although  it  may  well  be  highly 
correlated  with  it.  The  "general"  factor  is  general  only  in  the 
sanse  that  it  has  substantial  loadings  on  all  seven  variables.) 

The  Schmid-Leiman  factor  matrix  produces  the  same  reproduced  cor- 
relation matrix  as  the  orthogonal  two-factor  solution  does;  that 
is,  it  accounts  for  the  data  equally  well,  although  less  parsimoni- 
ously. The  Schmid-Leiman  "hierarchical"  procedure  has  been  too 
little  employed  in  factor-analytic  studies;  it  provides  one  way  of 
resolving  the  perennial  controversy  between  those  who  argue  for 
simple  structure,  correlated  factors  (when  necessary),  and  par- 
simony, and  those  who  argue  for  orthogonal  factors  because  of  their 
ease  of  Interpretation. 

In  general,  of  course,  the  problem  of  rotation  to  simple  struc- 
ture is  a very  tricky  one.  As  an  old  hand  in  factor  analysis,  I 
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Toble  1 


Reanr.lysis  of  Data  from 

a 

Study  by 

Jarmf n 

and  Das 

(1977) 

Eitlmeted  Correlation 

•^latrlx  for 

Pooled 

Groups 

(M  = 

180)® 

1 

2 

3 

4 

5 

6 7 

Raven's  Progressive  ’letrlces 

1 

1.000 

Figure  Copying 

2 

.503 

1.000 

.■■f'‘nory  for  design.s^ 

3 

.545 

.547 

1.000 

Serif  1 rf'call 

4 

.218 

.206 

.202  1 

.000 

Visual  short -tern  memory 

5 

.207 

.185 

.193 

.510 

1.000 

'^orci  reading  (speed 

6 

.206 

-.022 

.022 

.503 

.446 

1.000 

Auditory-visual  matching 

7 

.442 

.433 

.410 

.489 

.360 

.149  1.000 

Solutions  with  T^x)  Factors 

Common  Factor 
Orthogonal  Viarlmax 


Oblique 

Simple 

Structure 


Schmid-Lelrssn 
Hierarchical 
Or thogona 1 i za  t ion 


A 

B 

h^ 

A' 

B' 

G 

A" 

B" 

Raven's  Progressive  'iatrices 

1 

.672 

.178 

.483 

.615 

.083 

.447 

.527 

.071 

Figure  copying 

2 

.741 

.042 

.551 

.713 

-.062 

.418 

.611 

-.053 

Memory  for  designs^ 

3 

.748 

.058 

.563 

.716 

-.047 

.429 

.614 

-.040 

Serial  rnc.ill 

4 

.216 

.770 

.640 

.037 

.732 

.494 

.032 

.628 

Visual  short-term  memory 

5 

.189 

.640 

.445 

.040 

.608 

.415 

.034 

.521 

c 

Word  reading  (speed) 

6 

-.023 

.661 

.437 

-.171 

.658 

.312 

-.147 

.564 

Auditory-visual  matching 

7 

.558 

.384 

.459 

,457 

.303 

.487 

.39? 

.260 

A .974  -.139 

B -.225  .990 

R 

A*  1.000  .358 

B'  .358  1.000 


[Footnotes  for  Table  1] 
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These  correlations  were  estimated  by  pooling  correlation 

matrices  for  the  Low,  Normal,  and  High  IQ  groups  as  reproduced 

\ 

from  the  factor  matrices  presented  by  Jarman  and  Das,  using  also 
the  data  given  for  means  and  S.  D.'s  of  each  variable  for  each 
group . 

b 

This  variable  was  reflected  from  the  original  variable, 

which  was  in  terms  of  error  scores.  Here,  the  variable  may  be 

thought  of  as  number  correct, 
c 

This  variable  was  reflected  from  the  original  variable,  which 
was  in  terms  of  time  to  read  40  words. 


) 
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would  still  prefer  rotations  done  completely  by  hand,  i.e.,  by 
graphical  techniques,  but  I haven't  done  one  of  these  messy 
jobs  for  any  large  study  in  about  20  years,  now  that  analytic 
rotations  by  computer  are  available.  My  current  practice  is  to 
use  the  Kaiser  normal  varimax  method  to  produce  an  orthogonal 
solution,  followed  by  any  graphical  adjustments  to  obliqueness 
that  may  seem  desirable.  In  this  I often  use  a "semianaly tical" 
Procrustes  rotation  to  the  oblique  structure  suggested  by  the  pat- 
tern of  varimax  results,  using  a method  developed  by  Tucker  (1944). 
The  direct  oblimin  method  (Jennrich  & Sampson,  1966)  is  also  to  be 
recommended.  I have  not  taken  any  extensive  opportunity  to  experi- 
ment with  Joreskog's  methods  (1967  , 1970),  but  they  have  much  ap- 
peal because  they  Include  tests  of  statistical  significance. 

While  we  are  on  the  matter  of  rotation,  however,  let  me  mention 
some  possibilities  that  might  b..  considered.  One  of  the  more  in- 
teresting factor  analysis  studies  that  I have  encountered  was  that 
of  Underwood,  Boruch,  and  Malmi  ( 19  77).  These  investigators  factored 
correlations  of  22  scores  from  nine  verbal  learning  and  memory  tasks, 
hoping  to  find  factors  associated  with  particular  "attributes"  of 
memory  items  such  as  concreteness/abstractness,  meaningfulness,  and 
time.  In  this  they  were  largely  unsuccessful,  concluding  that  "as- 
sociative memory"  variance  was  so  prominent  as  to  swamp  any  effects 
of  memory  attributes,  also  pointing  out  that  apparently  subjects 
adapt  themselves  to  use  whatever  attributes  are  relevant  for  a par- 
ticular task.  Nevertheless,  the  final  factor  analysis  data  are  of 
interest,  even  though  the  five  factors  are  all  largely  task-specific. 
That  is,  there  is  a factor  that  loads  on  scores  from  paired-associate 


and  serial  learning  tasks,  one  that  loads  on  scores  from  free-recall 


tasks,  and  so  on 


The  N of  200  was  respectable,  and  the  factor 
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analysis  was  performed  In  a sophisticated  way,  using  Joreskog's 
(1967)  maximum  likelihood  method,  among  others.  If  one  plots 
the  factor  loadings  for  Factor  1 (the  "paired-associate"  factor) 
and  Factor  2 (the  " f ree- re  cal l"  factor)  that  are  shown  in  their 
orthogonal  factor  matrix  (Table  4,  p.  59),  an  obvious  oblique 
simple  structure  rotation  is  possible,  as  shown  in  Figure  1. 


Insert  Figure  1 about  here 

The  normals  to  the  two  hyperplanes  are  separated  by  an  angle  of 
about  138® , and  this  would  correspond  to  a correlation  of  about 
.743  between  rotated  primary  factors.  (I  ignore  any  other  pos- 
sible rotations.)  Thus  we  could  have  a strong  second-order 
factor  underlying  both  the  paired-associate  and  free-recall  scores; 
one  might  Interpret  it  as  an  "associative  memory"  factor  and  iden- 
tify it  with  the  associative  memory  factor  (often  symbolized  as 
Ma ; see  Harman,  Ekstrom,  & French,  1976)  that  has  been  found  by 
Thurstone  and  many  others  in  the  psychometric  tradition.  A Schmid- 
Lelman  orthogonalization  would  yield  this  "associative  memory" 
factor  as  a group  factor,  plus  separate  factors  for  the  paired  as- 
sociate and  free-recall  tasks.  In  this  case,  the  S chmid-Lelman 
procedure  would  be  neither  parsimonious  nor  very  informative. 
Another  kind  of  analysis  might  be  more  useful;  let  us  pass  a pri- 

1 

{ mary  vector  through  the  centroid  of  both  the  paired-associate  and 
free-recall  test  vectors,  letting  this  represent  an  associative 

j memory  factor,  underlying  these  two  types  of  tasks.  Orthogonal 

i 

I to  this  vector,  as  shown  in  Figure  2,  we  could  establish 


Insert  Figure  2 about  here 
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it  wholo,  pi'ovl  ill'll  f it  f tors  itro  iilwiiys  rot  it  toil  pnlrwlso  it  ml  ortho- 
Konii  1 1 y . ) 

In  I'ontInnInK  this  orltlijno  of  f a o | o r- iin  it  1 V t 1 o mot  hoilo  1 onv  , 

I will  rofor  to  vnrlons  si  nil  It'S  oiliof  I him  I hoso  ot  .1  nr  man  ami 
Has  ami  of  Umlorwooil  ami  his  oo  | I oaxnos . Cofialn  snhl  1 »'  mothoilo- 
I option  I prohli'ms  aro  11  Inst  rat  oil  In  stiullos  oo  ml  no  I oil  hv  tho  uronp 
at  tho  llnlvorsliy  ol  Wash  I iiKI  <'it  (Hunt,  Frost,  A l.nnnohor>t,  1 7 1 ; 
Hunt,  l.nnn  oh  o r p, , N l.owls,  I <7  / S 1 . 

( I'ho  pi  oh  lorn  of  o xpo  r 1 mon  t a 1 ilopomlonoo.  In  tho  th»orv  of 

mairloos,  It  Is  shown  that  If  any  varlahlt'  or  Ilnoar  oomhlnatlon  of 

varlahlos  oan  ho  porfi'otly  proilloioil  from  anoihor  varlahlo  or  Hnoar 
oomh I nation  ol  varlahlos,  Iho  matrix  of  oorrolatlons  amonp  tho  varl- 
ahlos Is  "sinpniar,"  with  no  invorso.  In  f ito  t o r-iinii  1 y I 1 o oompnta- 
tlons  with  mat  I loos  that  aro  sliiKnl-tr  or  app  r o x 1 ma  t o I y so,  tho  of- 
foot  ol  Iho  slnRiilarlty  can  show  np  as  spnrlons  oommon  factor  varl- 
anoo  (I.O.,  oommon  fnotor  varlanoo  that  wonlil  ho  nnli|no  varlanot'  11 
It  woro  not  for  tho  s 1 n Rii  1 a r 1 i I o s ) . In  many  oasos,  ono  flnils  oom- 
mon laotors  that  aro  assoolatoil  sololy  with  tho  sonroo  of  tho 


singularity,  but  a partially  spurious  c<'mmon  factor  could  be  one 
that  is  associated  with  the  variables  producing  a singularity 
plus  any  variables  that  are  substantially  correlated  with  them. 

In  the  light  of  these  considerations,  it  has  sometimes  been 
argued  that  one  should  avoid  any  potential  source  of  singularity 
(even  approximate)  at  all  costs,  even  to  the  extent  of  avoiding 
measuring  two  or  more  variables  from  the  same  experimental  task. 

Such  variables  are  said  to  be  experimentally  dependent.  In  the 
light  of  experience,  however,  I believe  the  rule  of  avoiding  mul- 
tiple measurements  from  a single  task  is  too  stringent.  There 
are  many  instances  where  one  can  derive  a number  of  logically  in- 
dependent variables  from  the  same  task.  By  "logically  independent," 
I mean  "conceivably  having  a correlation  of  zero."  For  example, 
the  rate  at  which  a task  is  performed  can  be  logically  independent 
of  the  accuracy  with  which  it  is  performed.  Of  course,  speed  and 
accuracy  may  in  fact  be  substantially  correlated,  either  positively 
or  negatively,  but  if  they  are  not  highly  correlated,  and  if  there 
are  independent  measures  of  each  of  them  from  a variety  of  other 
tasks,  the  risk  of  obtaining  spurious  common  factors  is  minimized. 

One  source  of  singularity,  however,  should  be  carefully  avoided 
the  use  of  sum  and  difference  scores  when  these  sum  and  difference 
scores  are  perfectly  predictable  from  other  variables  that  are  used 
in  the  correlation  matrix.  (The  lower  bound  communality  estimates 
for  such  variables  would  be  unity.)  It  is  an  egregious  error,  for 
example,  to  use  such  sets  of  variables  as  (A,  B,  A + B)  or  (A,  B, 
B-A) . Such  an  error  is  Illustrated  by  the  use  of  a series  of  part 
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total  scorei  but  not  both.  But  what  about  the  situation  in  which 
it  Is  desired  to  use  variables  (A,  B-A) ? This  is  exemplified  at 
least  twice  in  the  Hunt,  Lunneborg,  and  Lewis  (1975)  factor  anal- 
ysis of  psychometric  and  experimental  tasks.  One  pair  of  variables 
was  derived  from  a modified  Stroop  task:  "asterisk  reading  time," 
and  "color  name  minus  asterisk  reading  time."  Another  pair  of 
variables  was  derived  from  the  Posner  task:  "physical  match  time" 
and  "name  minus  physical  match  time."  Now,  the  variables  in  such 
pairs  are  logically  Independent,  and  they  could  be  completely  un- 
correlated in  the  data.  But  notice  that  if  the  basic  variables 
(which  might  be  symbolized  as  A and  B,  so  that  for  example  A = 
name  match  time,  B = physical  match  time)  are  uncorrelated,  there 
is  an  inevitable  negative  correlation  between  the  derived  variables 
A and  (B  - A)  that  will  produce  artifactual  common  factor  variance. 
For  the  general  case,  the  correlation  between  A and  (B  - A)  will  be 
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and  the  requirement  on  a zero  correlation  of  the  derived  variables 

is  that  r^g  = "^A^^B  ’ ^ requirement  that  would  not  ordinarily  be 

satisfied  in  practice.  In  fact,  the  correlation  between  the  derived 

variables  will  always  be  negative  if  r._  < ct./o„  ; thus,  the  correla- 

AB  A B 

tlon  depends  strongly  on  the  ratio  of  the  standard  deviations. 

(Under  non-linear  transformation,  such  as  the  logarithmic  or  recipro- 
cal transformations,  this  ratio  can  change  markedly.) 

It  will  be  noticed  in  the  Hunt  et  al.  (1975)  factor  matrix 
Just  mentioned  that  both  of  these  pairs  of  variables  have  negative 
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loadings  on  Factor  II;  In  each  case,  furthermore,  there  is  a 
high  loading  for  the  difference  variable  and  a lower  loading  for 
the  A-variable.  Undoubtedly  there  Is  some  spuriousness  In  these 
results,  but  It  Is  difficult  to  tell  how  much  there  was  because 
It  would  appear  that  the  difference  variables  were  correlated 
across  Independent  tasks.  My  recommendation  for  future  work  of 
this  type  Is  that  pairs  of  variables  that  Include  one  basic  vari- 
able and  one  derived  variable  should  be  measured  from  Independent 
tasks,  e.g.,  the  physical  match  score  from  one  series  of  trials 
and  the  name-physical  match  difference  from  another  series  of 
trails,  even  at  the  cost  of  lessened  reliability.  Even  so,  the 
difference  score  variables  might  have  been  correlated  over  tasks 
spuriously,  l.e.,  on  the  basis  of  an  overall  RT  factor.  The 
problem  Is  similar  to  that  encountered  In  the  Hock  (1973)  study 
mentioned  earlier.  A possible  solution  Is  to  compute  the  derived 
variables  In  terms  of  standardized  scores  of  basic  variables. 

(9)  The  design  of  a factor  analysis.  Ideally,  a set  of  vari- 
ables entered  Into  a factor  analysis  should  conform  to  a hypothesized 
structure  in  which  each  factor  has  at  least  three  or  four  signifi- 
cant loadings  that  are  not  accompanied  by  significant  loadings  on 
other  factors,  and  In  which  each  variable  has  a minimum  of  non-zero 
loadings — preferably,  only  one,  unless  the  variable  is  regarded  as 
Impure  ex  hypothes 1 . It  is  not  generally  good  science  to  factor- 
analyze  any  arbitrarily  selected  series  of  variables.  It  is  not 
necessarily  good  science  to  frame  a factor  analysis  according  to 
the  question  "To  what  extent  do  the  psychometric  and  the  Information- 
processing  variables  measure  the  same  abilities?"  To  this  extent 
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the Hunt  et  al.  studies  could  be  faulted.  In  these  studies  there 
Is  little  evidence  either  that  the  Information-processing  variables 
were  selected  specifically  to  coordinate  with  the  psychometric  tasks, 
or  conversely.  Furthermore,  the  specific  primary  factors  that  are 
known  to  be  measured  by  the  several  psychometric  tests  (Space, 

Number,  Verbal,  etc.)  were  not  separately  represented  by  adequate 
marker  tests  in  the  battery  design,  so  that  It  Is  not  surprising 
that  the  significant  loadings  for  the  psychometric  tests  came  out 
chiefly  on  a single  factor.  Factor  I (Hunt,  Lunneborg,  & Lewis, 

1975,  p.  222),  which  had  experimental  task  loadings  chiefly  on  scores 
derived  from  a complex  mental  arithmetic  task,  the  "Sunday  + Tuesday" 
task — surely  a task  that  would  have  counterparts  in  the  numerical 
and  verbal  reasoning  psychometric  tasks. 

In  fact,  the  irony  of  the  Hunt,  Lunneborg,  and  Lewis  study  Is 
that  although  it  claims  to  find  common  variance  between  the  psycho- 
metric tests  and  the  information-processing  tasks,  such  variance 
Is  only  weakly  apparent  in  the  factor-analytic  results.  In  general, 
the  two  sets  of  variables  have  significant  loadings  on  dlf f erent 
factors;  in  particular,  the  purely  "verbal"  tests  had  no  signifi- 
cant loadings  on  the  factors  that  were  chiefly  associated  with  the 
information-processing  tasks.  It  is  difficult  to  square  this  ob- 
servation with  the  generally  significant  contrasts  between  "high 
verbals"  and  "low  verbals"  that  are  reported  in  the  descriptions 
of  results  for  specific  tasks,  and  that  are  supposed  to  tell  us  "what 
it  means  to  be  high  verbal."  I can  possibly  reconcile  these  sets 
of  results  by  assuming  that  the  factor  analysis  was  not  done  by 
proper  common-factor  techniques,  and  not  carried  to  the  point  of 
oblique  rotation,  second-order  analysis,  or  hierarchical  analysis. 
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The  high-low  verbal  contrast  might  have  appeared  at  the  level  of 
a weak  but  significant  second-order  factor. 

If  Indeed  one  wants  to  ask  the  question  that  Hunt,  Lunneborg, 
and  Lewis  asked,  ”To  what  extent  do  the  psychometric  and  Information 
processing  tests  measure  the  same  abilities?",  there  are  variants 
of  factor  analysis,  and  other  multivariate  techniques,  that  would 
be  more  appropriate  than  classical  factor  analysis.  One  of  these 
is  Tucker's  (1958)  Interbattery  factor  analysis;  the  use  of  this 
method  is  well  illustrated  in  a study  reported  by  Hundal  and  Horn 
(1977).  They  administered  a series  of  "psychometric"  tests,  as 
one  battery,  and  a series  of  learning  and  memory  tests,  as  another 
battery,  to  265  14-year-old  school  children  in  India.  Tucker's 
method  was  used  to  determine  what  kinds  of  variance  were  common 
to  the  two  batteries.  They  identified  two  such  types  of  variance: 
a "Gf"  or  fluid  intelligence  factor  in  the  psychometric  tests  that 
was  related  to  the  primary  memory  storage  aspects  of  the  memory 
battery,  and  a "Gc"  or  crystallized  intelligence  factor  that 
was  more  related  to  those  aspects  of  the  memory  battery  that  relied 
on  what  they  regarded  as  a "secondary  acquisition  process." 

Canonical  correlation  analysis 

One  might  also  use  canonical  correlation  analysis  to  identify 
sources  of  variance  common  to  two  sets  of  measures.  In  effect, 
one  finds  linear  composites  in  one  set  of  measures,  say  the  psycho- 
metric tests,  that  correlate  optimally  with  linear  composites  of 
the  other  set  of  measures,  say  the  information-processing  tasks. 

This  technique  was  in  fact  used  by  Lunneborg  (1977),  but  I do  not 
recommend  it  for  this  purpose  because  canonical  weights  are  generally 
difficult  to  interpret.  The  technique  is  highly  subject  to  problems 
of  collinear Ity , and  canonical  weights  have  the  undesirable  sampling 
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characteristics  of  beta-weights  in  multiple-regression  analysis. 

1 

Indeedt  canonical  correlation  analysis  is  a generalization  of  { 

] 

multiple-regression  analysis.  If  canonical  correlation  analysis 
is  to  be  used  at  all,  it  should  be  followed  by  rotation  of  the 
canonical  variates  and  computation  of  correlations  between  each 
variable  and  the  canonical  variates  (see  Cliff  & Krus , 1976;  Wood 
& Erskine,  1976).  In  this  case,  however,  the  results  will  probably 
resemble  those  of  Tucker's  interbattery  factor  analysis,  which  could 
have  been  used  in  the  first  place. 

Multiple  regression  analysis 

Multiple  regression  of  course,  a popular  technique,  partlcu' 

larly  for  prediction  studies.  I believe,  however,  that  it  should 
be  used  only  very  cautiously  in  basic  studies  of  individual  differ- 
ences. The  problem  of  collinearity  among  predictor  variables  is 
especially  to  be  attended  to  In  the  interpretation  of  either  raw 
or  standardized  regression  weights.  This  problem  was  encountered, 
for  example,  by  Sternberg  (1977),  p.  219)  in  his  attempt  to  deter- 
mine "structural  regressions"  relating  "component  scores"  from 
analogical  reasoning  tasks  to  overall  scores  on  these  tasks  and 
reference  ability  measures.  If  multiple  correlation  techniques 
are  to  be  used,  I would  warn  particularly  against  the  popular  for- 
ward stepwise  solution,  which  can  be  very  misleading.  It  can  even 
fail  to  produce  the  optimal  subset  of  variables  for  predicting  a 
criterion  variable,  by  stopping  short  of  taking  account  of  some 
significant  comb Ina t ion  of  variables  at  a point  when  no  s ingle 
variable  adds  significant  variance.  I recommend  the  use  of  a com- 
plete regression  system,  followed  by  a backward  stepwise  elimination 
technique  if  perchance  one  wants  to  reduce  the  number  of  variables 
in  the  prediction.  With  care,  one  can  usually  make  some  acceptable 
interpretations  of  multiple  regression  weights. 


The  Identification  and  Investigation  of  Psychological  Processes 


Having  completed  my  catalogue  of  the  major  statistical  sins 
that  can  be  committed  in  individual  difference  studies,  let  me 
now  turn  to  the  more  psychological  side  of  the  question,  how  shall 
we  study  individual  differences? 

I said  earlier  in  this  paper  that  I believe  we  have  made  little 
progress,  thus  far,  in  identifying  psychological  processes,  at  least 
through  individual  difference  research.  In  most  of  the  individ- 
ual difference  research  I have  surveyed,  it  seems  that  whenever  one 
tries  to  find  a process,  one  really  finds  a trait.  That  is,  in 

( 

simple  terms,  one  sets  up  a study  involving  certain  tasks.  One  ; 

i 

notices  that  individuals  differ  in  their  success,  speed,  or  effi-  | 

ciency  in  performing  these  tasks.  Through  factor  analysis  or  other  ! 

techniques,  one  tries  to  pin  down  the  types  of  tasks,  or  components 

of  those  tasks,  in  which  the  individual  differences  can  be  observed, 

and  then  to  determine,  if  possible,  to  what  range  of  other  tasks 

(or  components  of  tasks)  these  individual  differences  generalize. 

Even  if  one  is  successful  in  such  a venture— and  there  are  relatively 
few  success  stories  — the  final  data  comprise  the  following:  a de- 
scription of  certain  tasks,  or  components  of  tasks,  and  a statement 
that  individuals  (in  defined  populations,  presumably)  exhibit  cha- 
racteristic differences  in  their  performance  of  these. tasks.  To 
the  extent  that  the  differences  are  truly  characteristic  of  the 
individuals  in  terms  of  relative  stability  and  permanence,  the  in- 
dividual differences  comprise  what  we  call  a trait.  Actually,  the 

1 

stability  of  individual  differences  in  most  of  the  information-pro-  j 

f 

cessing  traits  found  thus  far  has  hardly  been  investigated,  unless 


one  assumes  that  correlated  performanaes  spaced  a few  hours  or 
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days  apart  signify  a degree  of  stability.  How  much  do  we  know, 
for  example,  about  the  long-term  stability  of  simple  or  choice 
reaction  time  measures,  or  about  the  types  of  measures  taken, 
to  say.  In  the  Posner  physical  vs.  name  match  paradigm?  Not  much, 

I fear. 

The  Important  point,  however,  Is  that  Inferences  from  Individ' 
ual  difference  traits  concerning  underlying  psychological  processes, 
even  when  studied  In  an  Information-processing  mode,  seem  specula- 
tive at  best.  Perhaps  I am  asking  for  too  much,  or  wanting  too 
much.  In  a description  of  a psychological  process,  but  most  "de- 
scriptions" of  psychological  process  that  I have  seen  seem  to  boil 
down  to  descriptions  of  tasks--perhaps  in  terms  of  the  stimuli, 
the  Instructions,  and  the  responses,  with  speculations  about  what 
kinds  of  Information  are  Involved,  and  how  the  information  Is  trans- 
ferred and  manipulated.  There  Is  little  talk  of  different  strategies 
that  individuals  might  employ.  i 

Let  us  review  some  of  the  data  that  come  out  of  the  studies 
I have  referred  to  In  my  statistical  critique,  to  see  whether  my 
general  point  of  view  can  be  supported.  For  convenience,  I will 
refer  to  the  studies  more  or  leas  In  the  order  in  which  they  were 

discussed  earlier.  I 

i 

Cohen  and  Sandberg  (1977),  we  noted,  found  that  the  correlation 
between  IQ  and  memory  span  performance,  when  one  controlled  for  j 

rehearsal.  Interference,  chunking,  and  other  effects,  was  limited 
to  that  part  of  the  task  in  which  the  subject  repeats  the  last  few 
stimuli.  Even  though  there  was  evidence  (not  noted  by  these  authors) 
of  reliable  Individual  difference  variance  in  other  aspects  of  the 
task,  that  variance  was  not  associated  with  IQ.  High  IQ  children 
were,  however,  more  accurate  than  low  IQ  children  in  repeating  the 
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last  few  stimuli  of  a supraspan  digit  memory  task,  regardless  of 
the  modality  and  the  rate  of  presentation.  This  was  particularly 
true  when  the  Individuals  had  had  a prior  opportunity  to  practice 
a span  task  requiring  complete  recall.  These  results  were  In 
considerable  conflict  with  those  of  prior  studies,  which  had  lo- 
cated the  IQ/STM  correlation  mainly  In  rehearsal  effects.  Assum- 
ing that  the  Cohen  and  Sandberg  results  are  reliable,  what  can  we 
Infer  about  processes?  We  have  an  Individual  difference  variable 
that  Is  In  some  way  related  to  whatever  IQ  Is,  and  perhaps  the  re- 
sults tell  us  something  about  IQ,  but  they  do  not  directly  tell 
us  about  processes.  Cohen  and  Sandberg  themselves  argue  for  what 
they  call  an  "availability  explanation": 

In  a sequence  of  known  length.  Inclusion  In  the  rehearsal 
buffer  would  be  least  likely  for  the  final  Items,  so  that 
Individual  differences  In  decay  rate  would  exert  their 
greatest  influence  on  the  most  recent  items.  The  same  ar- 
gument can  be  made  for  the  final  items  in  the  running  memory 
sequences,  since  they  are  also  unlikely  to  be  maintained  in 
the  buffer,  (p.  552) 

The  "availability  explanation,"  then,  refers  to  an  assumption  of 
a "decay  rate"  for  recency  Items,  which,  according  to  the  theory 
the  authors  have  adapted  from  Atkinson  and  Shiffrin  (1968),  are 
"either  nonbuffer  items  from  STS  [short-term  store]  or  preatten- 
tional  items  from  SR  [sensory  register]"  (p.  537).  Presumably, 
further  study  would  be  required  to  decide  whether  the  decay  rate 
applies  to  the  former  or  to  the  latter,  or  to  both.  In  any  case, 
we  are  offered  only  speculation  about  what  processes  are  involved 
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In  the  found  IQ/STM  correlation;  the  only  solid  finding  is  that 
one  aspect  of  some  trait  called  IQ  is  the  efficiency  of  repeating 
the  last  few  digits  of  a supraspan  series.  I suppose  this  is 
progress,  however,  all  things  considered. 

The  series  of  alleged  findings  by  Hock  and  his  associates 
(e.g..  Hock,  1973;  Hock,  Gordon,  & Corcoran,  1976;  Hock  4 Ross, 

1975)  might  seem  a little  more  promising  in  allowing  inferences 
about  processes,  even  though  their  statistical  methodologies  and 
results  appear  open  to  question.  In  a variety  of  tasks,  these 
authors  claim  to  find  a contrast  between  what  they  call  "structural" 
processing  and  "analytic"  processing.  That  is,  in  the  same-different 
comparison  of  visual  displays  of  various  kinds  (dot  patterns,  pictures 
and  scenes,  degraded  alphabet  letters,  etc.),  some  people  are  thought 
to  make  point-by-point  or  "analytic"  comparisons,  while  others  make 
comparisons  based  on  a Gestalt  perception  of  the  total  stimulus, 
these  latter  being  called  "structural"  comparisons.  I will  not 
attempt  to  describe  here  the  exact  kinds  of  experimental  data  that 
are  offered  to  support  these  inferences;  I will  say  only  that  they 
are  highly  speculative,  based  on  certain  intuitions  about  how  certain 
stimulus  variables  might  have  an  effect  on  behavior.  For  example, 
certain  few  subjects  in  Hock's  (1973)  experiment  seemed  to  have 
longer  RTs  when  the  stimuli  were  asymmetric  or  rotated;  it  was 
inferred  that  they  were  perceiving  the  stimuli  "structurally"  and 
thus  the  asymmetry  and  rotation  Interfered  with  their  perceptions. 

The  remaining  subjects'  RTs  were  relatively  unaffected  by  rotation 
or  asymmetry;  it  was  inferred  that  they  were  making  polnt-by-polnt 
comparisons  of  the  stimuli  (whether  rotated  or  asymmetric  or  not) , 
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thus  doing  "analytic"  comparisons.  Nevertheless,  an  alternative 
interpretation  of  the  results  is  that  all  subjects  were  making 

point-by-polnt  comparisons,  but  that  some  subjects  were  less  able 
to  find  the  loci  of  these  comparisons  when  the  stimuli  were  rotated. 
Perhaps,  for  example,  these  subjects  would  be  found  to  be  low  on 
the  Spatial  Orientation  factor  Isolated  in  many  psychometric  in- 
vestigations. Hock  and  his  associates  made  no  attempt  to  identify 
an  Independent  variable  to  illuminate  their  results. 

I say  that  these  results  are  promising,  but  this  is  not  because 
of  anything  that  Hock  and  associates  have  done  with  their  experiments 
and  data.  "Structural"  and  "analytic"  processes  sound  like  distinctly 
different  mental  operations.  Let  us  suppose  that  they  are.  If  one 
could  arrange  a situation  where  a subject  could  be  successful  only 
if  he  made  a "structural"  comparison,  and  another  situation  where 
a subject  could  be  successful  only  by  making  an  "analytic"  compa- 
rison, and  if  some  independent  way  could  be  found  of  predicting  sub- 
jects' performances  in  the  two  situations,  we  might  have  a case  for 
the  reality  of  the  postulated  processes. 

Let  us  now  examine  the  theories  of  Jarman  and  Das  (1977)  con- 
cerning "simultaneous"  and  "successive"  syntheses.  Their  results 
concerning  differential  use  of  such  processes  in  groups  at  different 
IQ  levels  are  almost  completely  unconvincing  because  of  limitations 
in  their  methodology.  Leaving  this  aside,  however,  let  us  look  at 
the  reanalysls  of  their  results  that  I have  made  by  pooling  data 
from  the  three  IQ  strata  (Table  1) . If  we  consider  these  results 
from  the  standpoint  of  classical  factor-analytic  interpretations, 
it  appears  that  we  have  a general  factor  that  enters  all  the  tests, 
plus  two  group  factors,  A and  B.  Factor  A looks  like  a standard. 


- 35 


i 

i 

t 


ordinary  Space  factor;  that  is,  it  has  high  loadings  only  on  tasks  | 

in  which  the  subject  has  to  perceive,  remember,  or  otherwise  ma-  J 

I 

nipulate  visual  spatial  designs  Involving  geometric  figures. 

The  Figure  Copying  and  Memory  for  Designs  tests  contain  this 

feature,  as  does  also  the  Raven  Progressive  Matrices  (whatever  i 

i 

M 

else  it  may  involve) . Whatever  process  is  involved  in  such  a 
factor  can  only  be  described,  somewhat  tautologically,  as  "dealing 
with  spatial  designs."  Apparently  a characteristic  trait  of  in- 
dividuals is  their  ability  to  deal  with  spatial  designs,  and  little 
more  can  be  said  about  this  because  there  are  not  enough  variations 
among  the  teats  to  permit  further  analytic  interpretations.  We  can 
say,  however,  that  this  trait  does  not  extend  to  any  visual  display, 
because  it  does  not  extend,  for  example,  to  the  Visual  Short-Term 
Memory  task  that  requires  the  subject  to  remember  the  digits  that 
have  been  displayed  on  a five-section  grid  (essentially  a visual  i 

digit-span  test)  . It  is  a pure  inference  that  what  is  involved  i 

here  is  a "simultaneous  synthesis,"  i.e.  apprehending  parts  of  a 
display  simultaneously.  To  establish  that  simultaneous  synthesis 
is  a process  would  require  the  demonstration  that  it  operates  in  a 
variety  of  settings,  not  restricted  to  the  tasks  involving  visual 
patterns  and  designs  that  were  used  in  this  study.  Even  then,  use 
of  individual  differences  methodology  in  such  a demonstration  would 
permit  only  the  inference  that  people  differ  in  their  characteristic 
use  of  this  process  when  it  is  appropriate,  and  thus  we  would  end 

up  with  a statement  that  refers  more  to  a trait  than  it  does  to  a 

process. 

Similar  remarks  can  be  made  about  the  other  group  factor,  B, 
in  the  reanalyzed  Jarman  and  Das  data.  This  factor  looks  like  a 


memory  factor  that  crises'  when  the  serial  order  of  the  stimuli  is 
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important.  It  is  only  an  inference  that  some  special  "successive 
synthesis"  operation  is  involved. 

In  my  methodological  critique,  I next  considered  the  study 
by  Underwood,  Boruch,  and  Malmi  (1977),  which  I though  was  a very 
well  done  job.  From  an  information-processing  standpoint,  however, 
the  study  was  a failure;  what  these  investigators  came  out  with 
was  a series  of  traits,  not  processes.  That  is,  it  seems  that 
there  is  some  sort  of  "associative  memory"  trait  whereby  some 
people  are  better  than  others  at  paired-associate  and  free-recall 
tasks,  although  there  may  be  an  additional  trait  that  determines 
which  of  these  two  types  of  tasks  one  does  better  at  (at  least, 
this  is  what  my  reanalysis  suggests).  This  associative  memory 
trait,  I believe,  is  pretty  much  the  same  as  the  associative  memory 
factor  (f^)  that  has  been  identified  in  psychometric  researches  that 
are  too  numerous  to  list  here.  (See  Ekstrom,  1973,  for  a recent 
review.)  Also,  the  memory  span  factor  identified  in  the  Underwood 
et  al.  study  looks  to  be  the  same  as  the  or  Memory  Span  factor 

identified  previously.  The  fact  that  and  are  largely  in- 
dependent would  suggest,  however,  that  memory  span  ability  does 
not  operate  in  free  recall  tasks.  On  its  face,  this  conclusion 
sounds  rather  counterlntuitive--couldn ’ t one  do  a free-recall 
memory  task  as  a succession  of  memory  span  tasks?  But  the  results 
say  not;  if  memory  span  operations  are  attempted  in  a long  free-re- 
call task,  they  don’t  work,  perhaps  because  they  interfere  with 
each  other.  I am  not  enough  of  a specialist  in  memory  theory  to 
penetrate  deeply  into  this  question.  At  any  rate,  the  identifica- 
tion of  traits  of  associative  and  memory  span  abilities  tells  us 
very  little  about  the  processes  involved;  it  tells  us  only  that 

! different  processes  operate  in  different  task  settings,  and  this 
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1 n I o r mo  t t on  m I kU  t o on  o <'  i v nh  | y h o I p in  ti  h s I ^n  1 n p,  i lul  I v I il  no  1 n i i> 
il  I f I o r t'  t\  t t rt  8 k 8 «•  t t 1 ti  H f . 

U nilo  rwooil  ot  n I . mmio  n vtili.-ini  nitoinpi,  liowov«*r,  to  dcHlKn 
thoir  8tm\y  In  8ni’l\  a way  an  to  reveal  the  operation  of  whnt  they 
call  "memory  attrlhntes"  (Imanery,  aNsoelatlve,  aeonsile,  temporal, 
aft'eetlve,  and  Ireiineney).  TlM*y  tailed  to  find  1 art  ora  eorreapond- 
InR  to  these  attrlhntes,  ami  I re  lei  the  readi'r  to  their  report 
lor  their  apparent  ly  <inlte  reasonable  explanations  lor  this  failure. 
Hut  I would  also  nrtt<'  <'ons  I dera  i I on  of  the  possibility  that  their 
study  was  not  ad<'<inately  desluned  to  produce  (actors  lor  Imagery 
and  the  oth€*r  attrlhntes.  t'.onslder  the  Imagery  attrlhnle,  for  ex- 
ampli'.  llnless  I am  mistaken,  they  hail  too  few  measures  In  which  a 
cone  ret  e-ahs  t rac  t attrlhiiti*  would  have  opevateil  distinctively,  too 
few,  that  Is,  for  an  ade«inate  1 a c t o r- ana  I y t 1 c design.  (Kememher 
Thnrstone's  r ecomraomlat  1 on  that  there  he  at  least  three  variables 
to  r e p r e s e n l a p i' s t n I a t e tl  I a c t o i 1 n a s pure  a I r m as  possible.) 

An  alternative  design  that  might  have  been  consltleied  Is  the 
mn  1 1 1 t r a 1 1 - mn  I t 1 me  t hod  design  originated  by  t'ampbell  and  Klske 
(l‘)S‘);  see  also  recent  reflnem«*nts  proposeil  by  Kay  & Heeler,  l‘)7S). 
This  design  would  liave  reqnlred  a more  systematic  crossing  of  traits 
with  attrlbvites  (the  attributes  being  considered  analogous  to 
"me t hods " ) . 

Suppose*,  heiwc'ver.  Unde  rwo  oil  el  al.  had  been  successful  in  Iden- 
tifying an  "Imagery"  factor,  derived,  presumably,  from  sets  of 
variables  that  would  contrast  learning  of  concrete  stimuli  with 
learning  o I abstract  stimuli  In  d I I I e r e n t kinds  o 1 learning  tasks. 
Kssentlallv,  such  a factor  would  represent  a trait --the  extent  to 
which  a person  "uses"  imagery  (or  some  similar  process)  In  a learning 
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task  in  which  opportunity  to  use  that  process  was  offered.  The 
description  of  the  trait  would  refer  to  a process,  but  it  is  not  at 
all  clear  whether  that  process  had  been  correctly  identified.  One 
could  conceive  a number  of  processes  tliat  might  account  for  dif- 
ferences in  the  way  people  handle  concrete  words  vs.  abstract  words 
in  a learning  task.  For  example,  concrete  words  might  be  handled 
more  easily  than  abstract  words  by  attaching  attributes  to  them, 
and  people  might  differ  in  their  readiness  or  predisposition  to 
do  so.  (Experimentalists  often  attempt  to  control  such  variance 
by  equating  words  in  "associative  value,"  but  in  an  Individual 
differences  context  such  control  might  actually  be  counterproductive.) 

Now  consider  the  factor  interpretations  made  by  Hunt,  Lunneborg, 

& Lewis  (1975).  Factor  I was  loaded  with  a variety  of  psychometric 
test  scores  and  also  some  parameters  from  the  "Sunday  + Tuesday" 
mental  arithmetic  task.  They  called  it  "rapid  reasoning,"  since 
"it  is  characterized  by  tasks  which  involve  transformation  of  in- 
formation in  short  term  memory,  typically  in  a sequence  of  steps" 

(p.  222).  They  contrasted  Factor  I with  Factor  II,  which  "had  its 
highest  loadings  on  the  clerical  speed 'tests,  upon  scores  for  naming 
colors  less  scores  for  naming  asterisk  colors  in  the  Stroop  task, 
and  the  name  identity  minus  physical  identity  scores  in  the  Posner 
e t al . paradigm,"  tasks  that  were  "characterized  by  a requirement  that 

overlearned  codes  be  accessed,  but  not  by  the  (Factor  I)  requirement 

/ - 

that  the  codes  thus  accessed  be  transformed  in  any  way"  (pp.  222-223). 
Although  these  interpretations  are  plausible,  they  are  at  the  same 
time  problematical.  Undoubtedly  the  psychometric  tests  on  Factor  I 
Involve  reliance  on  the  presence  of  codes  in  long-term  memory 
( moanlngs  of  words  in  the  verbal  tests,  for  example).  A 
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more  sharply  focused  series  of  experimental  settings  would  be  re- 
quired to  demonstrate  that  the  factor  involves  transformatlors 
of  codes  in  STM.  The  interpretation  of  Factor  II  as  speed  in 
accessing  unt  r ans  formed  codes  in  STM  could  be  defended  only  if 
it  could  be  shown  that  it  is  irrelevant  whether  codes  in  LTM,  or 
transformed  codes  in  STM,  are  involved.  In  the  case  of  both 
Factors  I and  II,  the  factors  are  defined  by  the  individual  trait 
findings,  and  the  inferences  about  processes  rely  on  theories  about 
information-processing  operations  that  would  need  to  be  confirmed 
in  procedures  that  would  not  rely  on  individual  differences.  Per- 
haps Hunt  and  his  colleagues  would  argue  that  these  theories  have 
indeed  been  confirmed  in  various  investigations  that  do  not  rely 
on  individual  differences.  My  only  point  is  that  from  the  factor 
analysis  results  alone  we  do  not  have  a confirmation  of  those 
theories;  we  have  only  certain  plausible  descriptions  of  individual 
difference  traits  whose  appearance  is  elicited  by  certain  types  of 
tasks . 

Similar  remarks  can  be  made  about  the  interpretation  of  Factors 
III,  IV,  and  V in  the  Hunt,  Lunneborg,  and  Lewis  study  and  I shall 
not  pursue  this  line  of  discussion  further.  Let  me  dispel  any  pos- 
sible tensions  about  my  remarks  by  saying  that  I consider  the  Hunt, 
Lunneborg,  & Lewis  study  very  valuable.  Aside  from  the  methodologi- 
cal and  theoretical  limitations  to  which  it  is  subject,  it  is  an 
important  venture  into  a relatively  new  field  of  scientific  endeavor, 
and  it  should  inform  us  all  about  research  needs  and  possibilities 
in  this  field. 

Studies  by  R.  Sternberg  (1977)  and  J.  Frederlksen  (1978;  Note 
1)  move  us  into  an  almost  totally  new  methodology,  that  of  what 
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Sternberg  calls  "componential  analysis."  This  methodology  appears 
to  offer  a greater  possibility  of  isolating  processes  than  tra- 
ditional factor-analytic  methodology,  and  should  be  examined 
closely . 

Briefly,  Sternberg's  methodology  involves  arranging  experi- 
mental situations  that  systematically  vary  the  subject's  opportuni- 
ties to  engage  in  different  information-processing  tasks.  The 
verbal  analogies  task  that  Sternberg  has  chosen  to  study  is  par- 
ticularly well  suited  for  use  of  this  methodology.  Thus,  if  the 
verbal  analogies  task  is  symbolized  as  the  presentation  of  a series 
of  stimuli  that  have  relations  A ; B ::  C : [D],  where  [D]  repre- 

sents a number  of  alternative  stimuli,  only  some  of  which  exhibit 
"true"  analogies,  Sternberg  can  present  the  stimuli  in  different 
temporal  sequences  such  that  the  processing  times  that  a subject 
requires  at  different  stages  of  the  solution  can  be  determined. 

For  example,  one  can  present  only  the  stimuli  A and  B,  allowing 
subjects  to  process  those  stimuli  (determining  a possible  relation- 
ship) before  the  presentation  of  the  C and  D stimuli.  These  pro- 
cessing times  can  then  be  contrasted  with  those  required  if  three 
stimuli.  A,  B,  and  C,  are  presented  simultaneously,  and  the  para- 
meters of  the  processes  can  be  determined  on  the  basis  of  several 


alternative  models  of  these  processes.  Essentially  the  procedure 
I is  a very  sophisticated  application  of  the  Bonders  subtraction  method. 

Sternberg  has  presented  results  that  suggest  that  this  method 
; can  illuminate  the  description  and  explanation  of  the  individual 

1 differences  found  on  standard  psychometric  tests,  in  particular. 


the  verbal  analogies  test  that  often  appears  in  mental  abilities 
batteries.  I have  some  problems  with  his  methodology,  in  particular 
the  very  small  number  of  subjects  (for  example,  N ••  16  in  the 


"People  Piece"  experiment)  and  the  shakiness  of  multiple  regression 
methods  employed  with  such  a small  number  of  subjects.  Nevertheless, 
the  results  are  of  high  interest  and  are  suggestive,  and  if  one 
limits  oneself  to  the  results  intrinsic  to  the  experiments,  the 
fits  to  models  are  impressive. 

J.  Frederiksen ' s (1978;  Note  1)  methodology  is  highly  similar, 
except  that  he  has  been  concerned  with  component  processes  in  read- 
ing. Reading  does  not  easily  lend  itself  to  the  type  of  sequential 
stage  presentation  that  is  possible  in  the  case  of  Sternberg's 
verbal  analogies  task;  instead,  it  was  necessary  for  Frederiksen 
to  present  a number  of  different  tasks  that  represent,  according 
to  a theoretical  analysis  of  the  task  of  reading  (in  particular, 
word  identification  and  recognition) , the  several  phases  of  this 
process.  For  example,  it  is  assumed  that  there  is  a stage  of  Per- 
ceptual Encoding,  with  two  subphases.  Grapheme  Encoding  (access 
of  letter  codes)  and  Encoding  Multi-letter  Units.  The  processing 
times  for  these  phases  are  measured  from  two  tasks,  a Letter  Match- 
ing task  (analogous  to  the  Posner  task)  and  a Bigram  Identification 
Task.  Further  stages  are  Decoding,  with  two  subphases.  Phonemic 
Translation,  and  Articulatory  Programming  (since  an  oral  response 
is  required),  and  final  Lexical  Access  stage;  measures  of  these 
processes  are  taken  from  word  and  pseudoword  naming  tasks.  A 
maximum  likelihood  factor  analysis  of  eleven  measures  exhibited 
good  fit  to  the  five  hypothesized  sources  of  variance;  these  five 
sources  of  variance  were  found  to  be  somewhat  correlated,  but  it 
is  of  great  Interest  that,  for  N * 20,  these  five  factors  yielded 
high  multiple  correlations  with  scores  on  standardized  reading  tests 
that  Involved  sentence  comprehension  processes  that  were  not  at  all 
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tapped  by  the  experimental  tasks,  which  went  no  further  than  re- 
quiring word  identification.  The  multiple  regression  weights, 
producing  multiple  correlations  ranging  from  .53  (for  the  Gray 
Oral  Reading  Test)  to  1.00  (!,  for  the  Total  Score  on  the  Nelson- 
Denny  Reading  Test)  , seemed  to  indicate  that  the  multiletter 
encoding  and  the  articulatory  programming  measures  were  the  major 
predictors.  These  results  can  be  interpreted  as  suggesting  that 
good  readers  use  strategies  of  phonemic  decoding,  while  poor 
readers  recognize  words  more  on  the  basis  of  whole-word  appearance. 
They  need  to  be  cross-validated , however,  with  larger  sample  sizes. 

It  appears  to  me  that  in  both  the  Sternberg  and  Frederiksen 
studies,  a good  case  can  be  made  for  the  confident  identification 
of  psychological  processes.  The  theorized  stages  are  operation- 
alized by  arranging  the  experimental  tasks  in  such  a way  as  to 
permit  a subject  to  perform  only  on  the  basis  of  a definable  set 
of  processes.  For  example,  in  the  Sternberg  task  the  presentation 
of  only  the  A-stimulus  permits  only  an  encoding  operation;  the 
later  presentation  of  the  B and  C stimuli  permits  performance  of 
operations  that  Sternberg  describes  as  inference  and  mapping, 
but  not  application,  which  is  possible  only  when  the  D stimulus 
Is  supplied.  In  Frederiksen ' s work  it  is  clear  that  no  lexical 
access  operation  is  normally  possible  if  the  stimuli  are  only 
single  letters. 

Nevertheless,  in  both  Sternberg's  and  Frederiksen ' s work  the 
Identification  of  stages  is  almost  crucially  dependent  upon  the 
presence  of  individual  differences  that  are  specific  to  those 
stages.  This  is  quite  explicit  in  Frederiksen ' s work,  where  the 
confirmation  of  the  stage  analysis  is  based  on  a maximum  likelihood 
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factor  analysis  of  a correlation  matrix  of  individual  difference 
variables.  In  Sternberg's  work  a stage  could  in  principle  be 
identified  by  associating  it  with  additive  constants  in  which 
individual  differences  are  negligible,  but  that  seldom  happened 
in  Sternberg's  data,  if  at  all.  We  have  here,  then,  the  same 
kind  of  parallelism  between  traits  and  processes  that  we  have 
observed  in  other  types  of  methodologies. 

Now,  perhaps  this  is  the  way  things  are — the  way  world  is. 

That  is,  perhaps  processes  are  clearly  identifiable  only  through 
their  association  with  individual  differences,  and  perhaps  it  is 
inevitable  that  there  should  be  individual  differences  associated 
with  any  given  psychological  process.  (Surely  the  converse  of 
this  is  not  true.)  But  if  this  is  really  the  case,  it  presents 
a discouraging  prospect  for  any  efforts  to  modify  individual  dif- 
ferences, or  even  to  utilize  individual  differences  in  Instruction 
or  training  in  innovative  ways.  It  would  appear  that  we  would 
have  to  fall  back  on  the  old  routines  of  task  analysis,  and  selection 
of  individuals  for  training  or  assignment  in  terms  of  their  known 
characteristics  in  relation  to  our  task  analyses.  The  prospects 
for  meaningful  attribute-treatment  interactions  in  learning  pro- 
cesses would  also  appear  to  be  dim. 

I do  not  think  the  picture  is  really  as  dark  as  it  would  ap- 
pear to  be,  or  as  I have  presented  it.  I have  deliberately  pre- 
sented it  in  a rather  bleak  form  in  order  to  stimulate  thinking. 

There  are  a number  of  considerations  that  might  alleviate  the  sit- 
uation . 


i 


For  one  thing,  I have  painted  an  undeservedly  harsh  picture 
of  the  potentialities  of  psychometric  and  information-processing 
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research  for  the  study  of  psychological  processes.  Actually, 
much  of  this  research  lends  Itself  more  readily  to  solid  inferences 
about  processes  than  I have  led  the  reader  to  think.  Processes 
have  a distinct  relation  to  the  requirements  and  characteristics 
of  a particular  task;  just  as  in  the  Sternberg  and  Frederiksen 
studies,  the  tasks  studied  in  both  the  psychometric  and  the  In- 
formation-processing  literatures  have  characteristic  requirements 
and  constraints  that  enable  one  to  make  fairly  confident  inferences 
concerning  the  processes  that  can  and  cannot  occur  in  performing 
those  tasks.  For  example,  in  the  several  tasks  loaded  on  Factor 
A of  my  reanalyzed  factor  analysis  of  the  Jarman  and  Das  (1977) 
data  (Table  I) , it  is  obvious  that  some  kind  of  apprehension  and 
matching  of  spatial  forms  must  occur  for  a subject  to  be  success- 
ful in  performing  the  task.  At  the  same  time  it  is  obvious  that 
use  of  long-term  memory  for  retrieving  historical  dates,  say,  is 
Irrelevant  and  useless  in  performing  these  spatial  tasks.  Even  if 
individual  differences  are  inextricably  linked  with  processes, 
individual  difference  methodologies  should  enable  us  to  narrow 
down  the  kinds  of  processes  associated  with  particular  tasks,  and 
to  investigate  the  generality  of  those  processes  over  different 
tasks.  This  requires  only  a systematic  effort  to  vary  tasks  in 
such  a way  that  the  relevant  individual  difference  traits  and  the 
processes  asssoclated  with  them  are  adequately  defined  in  terms  of 
task  characteristics.  (This  is  what  factor  analysts,  in  fact,  have 
said  all  along;  unfortunately  there  are  few  examples  of  a systematic 
Series  of  factorial  investigations,  other  than  Guilford's  (1967; 
Guilford  & Hoepfner,  1971) perhaps , that  demonstrate  the  utility  of 
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this  Investigative  strategy.) 

Secondly,  it  is  possible  in  principle  to  distinguish  processes 
that  are  required  by  a task  from  processes  that  are  optional.  This 
distinction  is  seldom  observed  in  psychometric  individual  difference 
research,  and  it  appears  with  low  frequency  also  in  the  information- 
processing research  that  I have  surveyed.  In  psychometric  research, 
the  significant  loading  of  a variable  on  a process-related  factor 
tell  us  only  that  oji  the  average . subjects  tend  to  use  the  process 
that  is  associated  with  the  factor,  rather  than  some  other  process 
that  might  be  effective  in  performing  the  task,  if  indeed  there 
are  several  alternative  optional  processes.  Earlier,  I cited 
Thurstone’s  observation  that  factor  analysis  cannot  tell  us  which 
of  several  alternative  processes  might  be  used  by  a subject  in 
performing  a task.  In  information-processing  research,  there 
are  few  investigations  of  alternative  processes  in  performing 
tasks,  although  one  example  that  comes  to  mind  is  Groen  and  Park- 
man's  (1972)  demonstration  of  alternative  processes  (counting  vs. 
use  of  addition  facts)  in  children's  solution  of  arithmetic  prob- 
lems. It  may  be  useful  to  call  optional  processes  "strategies," 
restricting  use  of  the  term  process  for  those  that  are  required 
by  the  task.  It  was  with  this  distinction  in  mind  that  I (Carroll, 
1976)  analyzed  the  tasks  represented  by  the  French,  ERstrom  and 
Price  (1963)  Kit  of  Reference  Tests  for  Cognitive  Factors  in  terms 
of  operations  (processes  required  by  the  task)  and  strategies , al- 
though I may  have  failed  to  observe  the  distinction  with  sufficient 
rigor. 

It  also  occurs  to  me  to  mention  that  in  the  study  just  referred 
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to,  I was  careful  to  distinguish  between  contents  of  the  memory 
stores  that  are  operated  on  and  the  operations  and  strategies 
that  perform  the  processing.  In  varying  tasks  to  observe  the 
functioning  of  processes  and  strategies,  it  will  be  important 
to  control  or  otherwise  take  account  of  differential  contents 
of  memory  stores  that  may  be  involved  in  tasks.  Many  of  the 
factors  Isolated  in  traditional  psychometric  research  seem  to  be 
associated  with  particular  content  constellations,  i.e.  with  the 
presence  or  absence  of,  or  with  the  quantity  and  type  of,  informa- 
tion that  is  processed.  Information-processing  research  can 
similarly  fall  into  the  trap  of  failing  to  distinguish  between  pro- 
cesses and  the  content  of  the  information  that  is  processed. 

As  an  illustration  of  a promising  methodology  that  has  thus 
far  been  little  used  in  individual  difference  research  in  an  in- 
formation-processing mode,  I may  refer  to  a study  by  C.  Frederlksen 
(1969).  This  study  has  apparently  received  little  attention  among 
information-processing  theorists,  although  it  has  been  hailed  by 
Messlck  (1972,  p.  368)  as  a "milestone  study  in  [the]  multivariate 
experimental  probing  of  complex  learning  processes."  Frederlksen 
studied  college  students'  learning  of  a 60-word  list,  under  three 
experimental  conditions:  (a)  a standard  serial  anticipation  task, 
(b)  a "clustering"  task  in  which  the  words  were  artificially  pre- 
sented in  clusters  of  five,  and  (c)  a free-recall  task  in  which 
the  60  words  were  presented,  as  it  were,  all  at  once.  There  was 
a separate  group  of  AO  cases  assigned  to  each  experimental  condi- 
tion. The  response  data  consisted  of  the  scores  on  each  of  the 
18  trials  that  were  given  to  each  group;  these  learning  curve  data 


were,  however,  transformed  into  five  components  that  explained 
nearly  all  the  variance  of  the  scores  according  to  a principal- 
component  technique  for  such  data  devised  by  Tucker  (1966),  a 
technique  that  has  also  been  used  by  Lelcht  (1972)  in  the  analysis 
of  free-recall  data.  Immediately  after  the  learning  task,  Frederik- 
sen  administered  to  his  subjects  a questionnaire  on  the  strategies 
they  thought  they  employed  in  performing  the  learning;  these  data 
were  also  reduced  to  a series  of  five  components,  representing 
typical  strategies  such  as  "Organization  by  Grouping,"  "Active  Se- 
quential Organization,"  and  Modification  of  Strategies."  Frederik- 
sen  also  administered  his  subjects  a series  of  psychometric  tests 
that  yielded  scores  on  seven  of  the  factors  represented  in  the 
French,  Ekstrom,  and  Price  (1963)  Kit  of  Reference  Tests  for  Cog- 
nitive Factors.  Thus,  Frederiksen  had  three  sets  of  data:  the 
psychometric  tests,  the  strategy  scores,  and  the  learning  component 
scores.  Using  mainly  canonical  correlation  analysis,  Frederiksen 
determined  relationships  between  these  sets  of  data  in  each  of  the 
experimental  groups  pairwise,  i.e.,  between  the  ability  tests  and 
the  learning  components,  between  the  ability  tests  and  the  strategy 
components,  and  between  the  strategy  components  arid  the  learning 
components.  Although  there  were  some  significant  relationships, 
they  showed  no  very  clear  pattern  overall.  Different  experimental 
conditions  elicited  markedly  different  strategies,  but  the  effects 
of  the  strategies  showed  up  only  weakly  and  inconsistently  in  the 
learning  components.  In  the  main,  the  only  ability  factor  that 
showed  reasonably  consistent  and  substantial  relationships  with 
learning  component  data  was  the  associative  memory  factor  Ma ; but 
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then,  it  might  be  said  that  such  a relation  could  only  be  expected, 
since  the  memory  tasks  in  the  test  and  in  the  learning  trials  had 
major  similarities. 

This  study  by  Frederiksen  has  its  frustrating  aspects.  The 
sample  sizes  (40  in  each  group)  were  relatively  small,  and  it  is 
difficult  to  interpret  the  meaning  of  the  learning  curve  components. 
In  fact,  Frederiksen  suggested  that  "the  amount  of  information 
about  human  learning  obtainable  from  the  behavior  of  learning 
curves  may  be  limited,  and  that  precise  prediction  of  learning 
performance  curves  may  not  be  the  most  important  function  of  a 
learning  theory"  (p.  68)--a  sentiment  echoed  by  Leicht  (1972),  among 
others . 

Strangely,  Frederiksen  apparently  never  thought  to  put  together 
his  ability  factor  data  and  his  strategy  component  data  to  establish 
predictors  of  the  learning  component  data.  It  occurred  to  me  to  do 
this,  using  the  data  reported  in  his  monograph.  I hoped  to  find 
linear  combinations  of  ability  factor  scores  and  strategy  component 
scores  such  that  the  combination  would  predict  a learning  component 
score  significantly  better  than  the  use  of  either  alone.  I inves- 
tigated the  five  ability  scores  (^,  F^,  and  ^)  and  the  two 

strategy  components  (3,  Active  Sequential  Organization  and  4,  Order 
Preserving  Mnemonics)  that  seemed  to  show  the  stronger  and  more 
consistent  predictive  relations  with  learning  components,  and  put 
these  in  multiple  regressions  for  predicting  each  of  the  five  learn- 
ing components,  in  each  of  the  three  experimental  groups.  In  all, 
there  were  fifteen  multiple  regressions.  As  one  may  see,  this  was 
purely  a fishing  expedition.  But  one  can  enjoy  a fishing  expedition 


even  if  it  is  unproductive,  as  this  one  was.  About  the  closest  I 
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came  to  catching  the  kind  of  fish  I wanted  (one  where  an  ability 
factor  and  a strategy  component  would  each  have  a significant 

regression  weight)  was  in  the  case  of  the  Free-Recall  group,  where 
Strategy  4,  Order  Preserving  Mnemonics,  had  a clearly  significant 
weight  (t(32)=3.17,  p <.01),  but  factor  F[a,  Associative  Fluency, 
had  a weight  that  only  approached  significance  ( t ( 32)  “1 . 9 8 , p <•10'), 
in  the  prediction  of  Learning  Component  III.  If  this  finding  is 
regarded  as  having  significance,  it  would  mean  that  people  with 
high  Associative  Fluency  do  particularly  well  in  a certain  phase 
of  free  recall  learning  if  they  adopt  a strategy  of  using  "order 
preserving  mnemonics,"  such  as  trying  to  make  sentences  from  the 
words  presented,  rather  than  merely  attending  to  their  sounds. 
Actually,  in  this  group,  there  was  a negative  correlation  (r  = 

-.34)  between  ^a  and  the  use  of  this  strategy. 

Although  my  fishing  expedition  was  unsuccessful,  it  illustrates, 
as  does  the  Frederiksen  study  as  a whole,  the  potential  use  of  a 
methodology  whereby  information  about  both  abilities  and  strategies 
might  be  collected  to  predict  or  account  for  differences  in  task 
performance.  Possibly  one  condition  that  mitigated  against  obtain- 
ing significant  results  in  my  fishing  expedition  was  the  use  of  a 
purely  linear  model;  I was  of  course  limited  to  such  a model  by 
the  data  available  to  me  in  Frederiksen' s publication.  Some  sort 
of  non-linear  model  that  would  simultaneously  predict  the  probabili- 
ties of  using  alternative  strategies  on  the  basis  of  ability  scores 
and  predict  performance  data  on  the  basis  of  strategy  selection  and 
ability  scores  would  seem  to  be  desirable. 

Conclusion 

After  this  critical  examination  of  a range  of  studies  that 


employ  correlational  methodologies  in  the  study  of  individual 
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differences  in  information  processing,  these  methodologies, 
especially  factor  analysis  and  its  relatives,  can  be  judged  still 
viable  and  effective  procedures.  If  studies  are  designed  to  ex- 
ploit the  full  potentialities  of  these  methodologies,  and  if  the 
methodologies  are  properly  applied,  it  seems  possible  to  arrive 
at  reasonable  and  probably  confirmable  conclusions  concerning  the 
identification  and  description  of  a variety  of  cognitive  operations. 
Thus  far,  however,  many  errors  have  been  made  in  applying  these 
methodologies.  Even  when  they  have  been  correctly  applied,  their 
potentialities  have  not  been  fully  exploited  due  to  poor  or  inade- 
quate study  design. 

Results  thus  far  have  suggested  the  existence  of  a series  of 
cognitive  processes  that  show  correspondences  to  individual  dif- 
ference traits,  but  the  stability  and  the  generality  of  these  traits 
have  not  been  firmly  established.  There  has  been  little  attention 
to  the  possibility  that  individuals  can  use  alternative,  optional 
processes  or  strategies  in  test  performance,  and  to  the  possibility 
that  these  processes  or  strategies  are  amenable  to  manipulation 
through  variation  of  task  characteristics,  instruction,  practice, 
or  other  maneuvers. 

Although  the  state  of  the  art  in  individual  difference  research 
in  an  information-processing  mode  can  be  thought  of  as  little  more 
than  embryonic,  this  type  of  research  appears  to  have  a promising 
future  if  the  recommendations  made  in  this  review  can  be  followed, 
and  if  logistic  considerations  are  not  Insuperable. 
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CODE  44 

NATIONAL  NAVAL  MEDICAL  CENTER 
EETHESDA,  MD  20014 

1 Library 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

5 Commanding  Officer 

Naval  Research  Laboratory 
Code  2627 

Washington,  DC  20?90 

1 OFFICE  OF  CIVILIAN  PERSONNEL 
(CODE  26) 

DEPT.  OF  THE  NAVY 
WASHINGTON,  DC  20390 

1 JOHN  OLSEN 

CHIEF  OF  NAVAL  EDUCATION  4 
TRAINING  SUPPORT 
PENSACOLA,  FL  32509 

1 Office  of  Naval  Research 
Code  200 

Arlington,  VA  22217 

1 Scientific  Director 

Office  of  Naval  Research 
Scientific  Liaison  Group/Tokyo 
American  Embassy 
APO  San  Francisco,  CA  96503 


1 SCIENTIFIC  ADVISOR  TO  THE  CHIEF 
or  NAVAL  PERSONNEL 
NAVAL  BUREAU  OF  PERSONNEL  (PERS  OR) 
RM.  4410,  ARLINGTON  ANNEX 
WASHINGTON,  DC  20?70 

1 DR.  RICHARD  A.  POLLAK 

ACADEMIC  COMPUTING  CENTER 
U.S.  NAVAL  ACADEMY 
ANNAPCLIS,  MD  21402 

1 Mr.  Arnold  I.  Rubinstein 

Human  Resoureces  Program  Manager 
Kaval  Material  Comm-nd  (0344) 

Room  1044,  Crystal  Plaza  #5 
Washington,  DC  20360 

1 Dr.  Worth  Scanland 

Chief  of  Naval  Education  and  Training 
Code  N-5 

NAS,  Pensacola,  FL  32508 

1 A.  A.  SJOHOLM 

TECH.  SUPPORT,  CODE  201 
NAVY  PERSONNEL  R4  D CENTER 
SAN  DIEGO,  CA  92152 

1 Mr . Robert  Smith 

Office  of  Chief  of  Naval  Operations 
OP-987E 

Washington,  DC  20350 

1 CDR  Charles  J.  Theisen,  JR.  MSC,  USN 
Head  Human  Factors  Engineering  Div. 
Naval  Air  Development  Center 
Warminster,  PA  18974 

1 DR.  MARTIN  F.  WISKOFF 

NAVY  PERSONNEL  Ri  D CENTER 
SAN  DIEGO,  CA.  92152 
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Army 


1 HQ  USAREUE  & 7th  Army 
ODCSOPS 

USAAREUE  Dirretor  of  GED 
APO  New  York  0940? 

1 DR.  JAMES  BAKER 

U.S.  ARMY  RESEARCH  INSTITUTE 
5001  EISENHOWER  AVENUE 
ALEXANDRIA,  VA  2233? 

1 DR.  RALPH  CANTER 

U.S.  ARMY  RESEARCH  INSTITUTE 
5001  EISENHOWER  AVENUE 
ALEXANDRIA,  VA  22333 

1 DR.  RALPH  DUSEK 

U.S.  ARMY  RESEARCH  INSTITUTE 
5001  EISENHOWER  AVENUE 
ALEXANDRIA,  VA  22333 

1 DR.  FRANK  J.  HARRIS 

U.S.  ARMY  RESEARCH  INSTITUTE 
5001  EISENHOWER  AVENUE 
ALEXANDRIA,  VA  22??3 

1 Dr.  Milton  S.  Katz 

Individual  Training  & Skill 
Evaluation  Technical  Area 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1 Dr.  J.  E.  Uhlaner 

Chief  Psychologist,  US  Army 
Army  Research  Institute 
6933  Hector  Road 
McLean,  VA  22101 

1 Dr.  Joseph  Ward 

U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 


Air  Force 


1 Air  Force  Human  Resources  Lab 
AFHRL/PED 

Brooks  AFB,  TX  78235 

1 Air  University  Library 
AUL/LSE  76/443 
Maxwell  AFB,  AL  36112 

1 DR.  G.  A.  ECKSTRAND 
AFHRL/AS 

WRIGHT-PATTERSON  AFB,  OH  45433 

1 Dr.  Alfred  R.  Fregly 
AFOSR/NL,  Bldg.  410 
Bolling  AFB,  DC  20332 

1 CDR.  MERCER 

CNET  LIAISON  OFFICER 
AFHRL/FLYING  TRAINING  DIV. 
WILLIAMS  AFB,  AZ  85224 

1 Dr.  Ross  L.  Morgan  (AFHRL/ASR) 
Wright  -Patterson  AFB 
Ohio  45433 

1 Personnel  Analysis  Division 
HQ  USAF/DPXXA 
Washington , DC  20330 

1 Dr.  Marty  Rookway  (AFHRL/TT) 

Lowry  AFB 
Colorado  80230 

1 Major  Wayne  S.  Sellman 
Chief,  Personnel  Testing 
AFMPC/DPMYPT 
Randolph  AFB,  TX  78148 

1 Brian  K.  Waters,  Maj.,  USAF 

Chief,  Instructional  Tech.  Branch 
AFHRL 

Lowry  AFB,  CO  80230 


1 Director,  Office  of  Manpower  Utilization  1 MR.  JOSEPH  J.  COWAN,  CHIEF 

HC,  Marine  Corps  (MPU)  P.SYCHOLOGICAL  RESEARCH  (G-P-1/62) 

BCB,  Bldg.  2009  U.S.  COAST  GUARD  HO 

Ouantico,  VA  221  ?U  WASHINGTON,  DC  20590 

1 DR.  A.L.  SLAFKOSKY 

SCIENTIFIC  ADVISOR  (CODE  RD-1) 

HO,  U.S.  MARINE  CORPS 
WASHINGTON,  DC  20?80 


Oth“r  DoD 


12  Def»»nsr  Documentation  Center 
Cp.meron  Station,  Bldg.  5 
Alexandria,  VA  2231** 

Attn:  TC 

1 Military  Assistant  for  Human  Resources 
Office  of  the  Director  of  Defense 
Research  4 Engineering 
Room  3D129,  the  Pentagon 
Washington,  DC  20301 

1 Dr.  Harold  F.  O’Neil,  Jr. 

Advanced  Research  Projects  Agency 
Cybernetics  Technology,  Rra.  623 
1i»00  Wilson  Blvd. 

Arlington,  VA  22209 

1 Director,  Research  & Data 

OSD/MR A 4L  (Rffl.  3B919) 

The  Pentagon 
Washington,  DC  20301 

1 Mr.  Fredrick  W.  Suffa 

MPP  (A4R) 

{ 2B269 

i Pentagon 

i Washington,  D.C.  20301 

i 

1 1 DR.  ROBERT  YOUNG 

I ADVANCED  RESEARCH  PROJECTS  AGENCY 

inoo  WILSON  BLVD. 

ARLINGTON,  VA  22209 
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Civil  Govt  i 


1 Dr.  William  Gorham,  Director 
Personnel  R4D  Center 
U.S.  Civil  Service  Commission 
1900  E Street  NW 
Washington,  DC  20M15 

1 Dr.  Andrew  R.  Molnar 
Science  Education  Dev. 

and  Research  i 

National  Science  Foundation  | 

Washington,  DC  20550  | 

i 

1 Dr.  H.  Wallace  Sinaiko,  Director  I 

Manpower  Research  & Advisory  Service  i 

Smithsonian  Institution  ] 

801  N.  Pitt  Street 
Alexandria,  VA  2231^ 

1 Dr.  Thomas  G.  Sticht 
Basic  Skills  Program 
National  Institute  of  Education 
1200  19th  Street  NW 
Washington,  DC  20208 

1 Robert  W.  Stump 

Education  4 Work  Group 
National  Institute  of  Education 
1200  19th  Street  NW 
Washington,  DC  20208 

1 Dr.  Vern  W.  Urry 

Personnel  R4D  Center 
U.S.  Civil  Service  Commission 
1900  E Street  NW 
Washington,  DC  20415 

1 C.S.  WINIEWICZ 

U.S.  CIVIL  SERVICE  COMMISSION 
REGIONAL  PSYCHOLOGIST 
230  S.  DEARBORN  STREET 
CHICAGO,  IL  60604 

1 Dr.  Joseph  L.  Young,  Director 

Memory  4 Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 
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Non  Govt 


1 PROF.  EARL  A.  ALLUISl 
DEPT.  OF  PSYCHOLOGY 
CODE  287 

OLD  DOMINION  UNIVERSITY 
NORFOLK,  VA  2?508 

1 Dr.  John  R.  Anderson 
Dept,  of  Psychology 
Yal*^  University 
New  Haven,  CT  06520 

1 DR.  MICHAEL  ATWOOD 

SCIENCE  APPLICATIONS  INSTITUTE 
UO  DENVER  TECH.  CENTER  WEST 
7935  E.  PRENTICE  AVENUE 
ENGLEWOOD,  CO  80110 

1 MR,  SAMUEL  BALL 

EDUCATIONAL  TESTING  SERVICE 
PRINCETON,  NJ  085‘t0 

1 Dr.  Gerald  V.  Barrett 
Dept,  of  Psychology 
University  of  Akron 
Akron,  OH  44325 

1 Dr.  Nicholas  A.  Bond 
Dept,  of  Psychology 
Sacramento  State  College 
600  Jay  Street 
Sacramento,  CA  95819 

1 Dr.  John  Seeley  Brown 

Bolt  Bcranek  & Newman,  Inc. 

50  :;oulton  Street 
CamVidge,  MA  021^8 

1 Dr.  Kenneth  E.  Clark 

College  of  Arts  h Sciences 
University  of  Rochester 
River  Campus  Station 
Rochester,  NY  14627 

1 Dr.  Norman  Cliff 
Dept,  of  Psychology 
Univ.  of  So.  California 
University  Park 
Los  Angeles,  CA  90007 


Non  Govt  ] 

I 

1 Dr.  Allan  M.  Collins 

bolt  Bcranek  & Newman,  Inc. 

50  Moulton  Street 
Cambridge,  Ma  02138 

1 Dr.  John  J.  Collins 
Essex  Corporation 

201  N.  Fairfax  Street 
Alexandria,  VA  22314 

1 Dr.  Meredith  Crawford 
5605  Montgomery  Street 
Chevy  Chase,  MD  20015 

1 Dr.  Donald  Dansereau 
Dept,  of  Psychology 
Texas  Christian  University 
Fort  Worth,  TX  76129 

1 DR.  RENE  V.  DAWIS 
DEPT.  OF  PSYCHOLOGY 
UNIV.  OF  MINNESOTA 
75  E.  RIVER  RD. 

MINNEAPOLIS,  MN  55455 

1 Dr,  Ruth  Day 

Center  for  Advanced  Study 
in  Behavioral  Sciences 

202  Junipero  Serra  Blvd. 

Stanford,  CA  94305 

1 MAJOR  I.  N.  EVONIC 

CANADIAN  FORCES  PERS.  APPLIED  RESEARCH 
1107  AVENUE  ROAD 
TORONTO,  ONTARIO,  CANADA 

1 Dr.  Richard  L.  Ferguson 

The  American  College  Testing  Program 
P.O.  Box  168 
Iowa  City,  I A 52240 

1 Dr.  Victor  Fields 
Dept,  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 
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Non  Govt 


Non  Govt 


1 Dr.  Edwin  A.  Fleishmsn  1 

Advanced  Research  Resources  Organ. 

8555  Sixteenth  Street 
Silver  Spring,  MD  20910 

1 Dr.  John  R.  Frederiksen  1 

Bolt  Eeranek  A Newman 
SO  Moulton  Street 
Cambridge,  MA  02138 


Dr.  Arnold  F.  Kanarick  i 

Honeywell,  Inc.  i 

2600  Ridgeway  Pkwy 
Minneapolis,  MN  55*H3 

Dr.  Roger  A.  Kaufman 
203  Dodd  Hall 
Florida  State  Univ. 

Tallahassee,  FL  32306 


DR.  ROBERT  GLASER 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
3939  O’HARA  STREET 
PITTSBURGH,  PA  15213 

1 DR.  JAMES  G.  GREENO 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
3939  O'HARA  STREET 
PITTSBURGH,  PA  15213 

1 Dr.  Richard  S.  Hatch 

Derision  Systems  Assoc.,  Inc. 
350  Fortune  Terrace 
Rockville,  MD  2085*1 

1 Dr.  Barbara  Hayes-Roth 
The  Rand  Corporation 
1700  Main  Street 
Santa  Monica,  CA  90*106 

1 Library 

HumRRO/Western  Division 
27857  Berwick  Drive 
Carmel,  CA  93921 

1 Dr.  Earl  Hunt 

Dept,  of  Psychology 
University  of  Washington 
Seattle,  WA  98105 

1 DR.  LAWRENCE  B.  JOHNSON 

LAWRENCE  JOHNSON  & ASSOC.,  INC 
SUITE  502 
2001  S STREET  NW 
WASHINGTON,  DC  20009 


1 Dr.  Steven  W.  Keele 
Dept,  of  Psychology 
University  of  Oregon 
Eugene,  OR  97*103 

1 Dr.  Ezra  S.  Krendel 
Wharton  School, DH/DC 
University  of  Pennsylvania 
Philadelphia,  PA  1917*1 

1 LCOL.  C.R.J.  LAFLEUR 
PERSONNEL  APPLIED  RESEARCH 
NATIONAL  DEFENSE  HQS 
101  COLONEL  BY  DRIVE 
OTTAWA,  CANADA  K1A  0K2 

1 Dr.  Frederick  M.  Lord 

Educational  Testing  Service 
Princeton,  NJ  085*10 

1 Dr.  Robert  R.  Mackie 

Human  Factors  Research,  Inc. 
6780  Cortona  Drive 
Santa  Barbara  Research  Pk. 
Goleta,  CA  93017 

1 Dr.  William  C.  Mann 

USC-In formation  Sciences  Inst 
*1676  Admiralty  Way 
Marina  del  Rey,  CA  90291 

1 Dr.  Richard  B.  Millward 
Dept,  of  Psychology 
Hunter  Lab. 

Brown  University  - 
Providence,  RI  82912 


FAGK 


Non  Govl 


II  Dr.  Domld  A Normon 

Drpt . of  Psychology  C-OOi} 

Unlv.  of  Cnllfornlo,  Diego 
; Ln  Jolla,  CA  0209 

I 

1 Dr.  Melvin  R.  Novick 
Iowa  Testing  Progrnms 
University  of  lovn 
Iowa  City,  lA  ‘i22‘*2 

1 Dr.  Jcs.so  Orlansky 

Institute  for  Dofrnsr  Analysis 
^00  Army  Navy  Drive 
Arlington,  VA  22202 

1 Mr.  A.  J.  Pesch,  President 
Koleoteoti  Associates,  Inc. 

P.  0.  hox  178 
N.  Stonlngton,  CT  Ob^SO 

1 MR.  I.U1G1  PETRULLO 

2lm  N.  KDGEWOOD  .STREET 
ARLINGTON,  VA  22207 

1 DR.  i'TEVEN  M.  PINE 
NhbO  ELLIOTT  HALL 
UNIVERSITY  OE  MINNESOTA 
Ab  E.  RIVER  ROAD 
MINNEAPOLIS,  MN  'isns'" 

1 DR.  PETER  POLSON 
DEPT.  OE  PSYCHOLOGY 
UNIVERSITY  UE  COLORADO 
BOULDER,  CO  HO ^02 

1 Dr.  Erank  Pratyncr 

Cntr.  for  Vocation'll  Education 
Ohio  State  University 
I960  Kenny  Road 
Columbus,  OH  MS210 

1 DR.  DIANE  M.  RAMSEY-KLEE 

H-K  RESEARCH  SY.STEM  DESIGN 
mi  RIDGEMONT  DRIVE 
MALIBU,  CA  9026S 


Non  Govt 


1 MlN.  RET.  M.  RAUCH 
P 11  *1 

BUNDESMINISTERIUM  DER  VERTEIDIGUNG 

POSTEACH  161 

S-i  BONN  1,  GERMANY 

1 Dr.  Mark  D.  Rcckase 

Educ.atlcnal  Psychology  Dept. 
University  of  Missouri-Columbla 
12  Hill  Hall 
Columbl.a,  MO  6S201 

1 Dr.  Joseph  W.  Rigney 
Unlv.  of  So.  C.allfornin 
Behavioral  Technology  L.abs 
1717  South  Hope  Street 
Los  Angeles,  CA  90007 

1 Dr.  Andrew  M.  Rose 

Am''rlcan  Institutes  for  Resesreh 
loss  Thomas  Jefferson  St.  NW 
Wa.ahlngton , DC  20007 

1 Dr.  Leonard  L.  Rosenbaum,  Chairman 
Department  of  Psychology 
Montgomery  College 
Rockville,  MD  20HS0 

1 PROE.  EUMIKO  SAMEJIMA 
DEPT.  CE  PSYCHOLOGY 
UNIVERSITY  OE  TENNESSEE 
KNOXVILLE,  TN  ?7916 

1 DR.  WALTER  .SCHNEIDER 
DEPT.  OE  PSYCHOLOGY 
UNIVERSITY  OF  ILLINOIS 
CHAMPAIGN,  IL  61820 

1 DR.  ROBERT  J.  SEIDEL 

INSTRUCTIONAL  TECHNOLOGY  GROUP 
HUMRRO 

’00  N.  WASHINGTON  ST. 

ALEXANDRIA,  VA  22? IH 

1 Dr.  Richard  Snow 
.School  of  Education 
Stanford  University 
Stanford.  CA  -^OS 
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1 Dr . Pobort  Stornbern 
Dept,  of  Psychology 
Yslo  University 
Box  11A,  Ynle  Station 
New  Haven,  CT  065P0 

1 DR.  ALBERT  STEVENS 

BOLT  BERANEK  & NEWMAN,  INC. 

50  MOULTON  STREET 
CAMBRIDGE,  MA  02136 

1 DR.  PATRICK  SUPPES 

INSTITUTE  FOB  MATHEMATICAL  STUDIES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD,  CA  9**?05 

1 Dr.  Kikumi  Tatsuoka 

Computer  Based  Education  Research 
Laboratory 

252  Engineering  Research  Laboratory 
University  of  Illinois 
Urbana,  IL  61801 

1 DR.  PERRY  THORNDYKE 
THE  RAND  CORPORATION 
1/00  MAIN  STREET 
SANTA  MONICA,  CA  90*106 

1 Dr.  Benton  J.  Underwood 
Dept,  of  Psychology 
Northwestern  University 
Evanston,  IL  60201 

1 DR.  THOMAS  WALLSTEN 

PSYCHOMETRIC  LABORATORY 
DAVIE  HALL  OipA 
UNIVERSITY  OF  NORTH  CAROLINA 
CHAPEL  HILL,  NC  2751** 

1 Dr . John  Wannous 

Department  of  Management 
Michigan  University 
East  Lansing,  Ml  **882** 

1 Dr.  Claire  E.  Weinstein 

Educational  Psychology  Dept. 

Univ.  of  Texas  at  Austin 
Austin,  TX  78712 


Non  Govt 


1 Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55**55 

1 DR.  KEITH  WESCOUBT 

INSTITUTE  FOR  MATHEMATICAL  STUf'ES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD,  CA  9**305 

1 Dr.  Anita  West 

Denver  Research  Institute 
University  of  Denver 
Denver,  CO  80201 

1 DR.  SUSAN  E.  WHITELY 
PSYCHOLOGY  DEPARTMENT 
UNIVERSITY  OF  KANSAS 
LAWRENCE,  KANSAS  660**** 


